Sep 19 10:36:30.390559 ip-10-0-142-165 systemd[1]: Starting Kubernetes Kubelet...
Sep 19 10:36:30.766409 ip-10-0-142-165 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 10:36:30.766409 ip-10-0-142-165 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 19 10:36:30.766409 ip-10-0-142-165 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 10:36:30.766409 ip-10-0-142-165 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 19 10:36:30.766409 ip-10-0-142-165 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 10:36:30.767089 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.766429    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 19 10:36:30.771012 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.770987    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 10:36:30.771012 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771012    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 10:36:30.771096 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771016    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 10:36:30.771096 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771020    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 10:36:30.771096 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771023    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 10:36:30.771096 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771027    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 10:36:30.771096 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771030    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 10:36:30.771096 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771033    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 10:36:30.771096 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771036    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 10:36:30.771096 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771059    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 10:36:30.771096 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771065    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 10:36:30.771096 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771071    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 10:36:30.771096 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771075    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 10:36:30.771096 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771078    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 10:36:30.771096 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771081    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 10:36:30.771096 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771084    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 10:36:30.771096 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771086    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 10:36:30.771096 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771089    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 10:36:30.771096 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771092    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 10:36:30.771096 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771095    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 10:36:30.771096 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771098    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771101    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771104    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771107    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771117    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771120    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771124    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771127    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771130    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771133    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771136    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771139    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771141    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771144    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771147    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771150    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771154    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771157    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771159    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771162    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 10:36:30.771550 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771165    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771167    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771170    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771173    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771178    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771182    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771185    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771188    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771190    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771193    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771195    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771198    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771201    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771203    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771207    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771210    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771212    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771215    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771218    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771220    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 10:36:30.772065 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771223    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771226    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771229    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771231    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771234    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771236    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771239    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771242    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771245    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771247    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771250    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771253    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771255    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771258    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771261    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771267    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771270    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771273    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771276    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771281    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 10:36:30.772559 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771283    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771286    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771289    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771291    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771294    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771297    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771301    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771304    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771307    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771309    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771312    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771314    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771317    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771320    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771323    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771325    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771328    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771331    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.771334    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772079    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 10:36:30.773030 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772087    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 10:36:30.773522 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772091    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 10:36:30.773522 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772094    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 10:36:30.773522 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772097    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 10:36:30.773522 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772100    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 10:36:30.773522 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772103    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 10:36:30.773522 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772106    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 10:36:30.773522 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772109    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 10:36:30.773522 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772114    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 10:36:30.773522 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772119    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 10:36:30.773522 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772122    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 10:36:30.773522 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772125    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 10:36:30.773522 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772128    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 10:36:30.773522 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772130    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 10:36:30.773522 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772133    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 10:36:30.773522 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772136    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 10:36:30.773522 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772139    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 10:36:30.773522 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772143    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 10:36:30.773522 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772148    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 10:36:30.773522 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772151    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772154    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772157    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772160    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772163    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772166    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772169    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772172    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772174    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772178    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772181    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772183    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772186    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772189    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772192    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772195    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772197    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772200    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772202    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772205    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 10:36:30.774036 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772207    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772210    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772213    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772216    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772219    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772221    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772224    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772226    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772229    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772232    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772235    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772239    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772242    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772245    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772248    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772250    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772253    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772256    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772258    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772261    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 10:36:30.774656 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772264    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 10:36:30.775169 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772266    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 10:36:30.775169 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772269    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 10:36:30.775169 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772271    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 10:36:30.775169 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772274    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 10:36:30.775169 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772277    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 10:36:30.775169 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772279    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 10:36:30.775169 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772283    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 10:36:30.775169 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772287    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 10:36:30.775169 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772290    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 10:36:30.775169 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772293    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 10:36:30.775169 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772295    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 10:36:30.775169 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772298    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 10:36:30.775169 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772301    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 10:36:30.775169 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772304    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 10:36:30.775169 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772307    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 10:36:30.775169 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772310    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 10:36:30.775169 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772313    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 10:36:30.775169 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772316    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 10:36:30.775169 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772318    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772321    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772323    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772326    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772329    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772332    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772335    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772337    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772340    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772342    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772345    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772348    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772350    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772353    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772356    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772358    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772361    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772364    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772366    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772369    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 10:36:30.775622 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772797    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772802    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772805    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772809    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772811    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772815    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772818    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772821    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772829    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772832    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772835    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772838    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772840    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772843    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772847    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772850    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772852    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772855    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772858    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772860    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 10:36:30.776118 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772863    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 10:36:30.776618 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772866    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 10:36:30.776618 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772868    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 10:36:30.776618 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772871    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 10:36:30.776618 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772875    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 10:36:30.776618 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772879    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 10:36:30.776618 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772884    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 10:36:30.776618 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772888    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 10:36:30.776618 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772891    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 10:36:30.776618 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772894    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 10:36:30.776618 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772897    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 10:36:30.776618 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772901    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 10:36:30.776618 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772904    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 10:36:30.776618 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772907    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 10:36:30.776618 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772910    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 10:36:30.776618 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772913    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 10:36:30.776618 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772915    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 10:36:30.776618 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772918    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 10:36:30.776618 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772922    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 10:36:30.776618 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772924    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772927    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772936    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772938    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772941    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772944    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772947    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772950    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772953    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772955    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772958    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772961    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772963    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772966    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772969    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772971    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772974    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772977    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772980    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772982    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 10:36:30.777097 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772985    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772988    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772990    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772993    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772996    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.772998    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773002    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773005    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773007    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773010    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773012    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773017    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773021    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773024    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773027    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773030    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773034    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773036    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773055    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773058    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 10:36:30.777574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773061    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773064    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773066    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773069    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773072    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773075    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773077    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773080    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773083    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773086    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773089    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773091    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773094    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773096    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773099    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773101    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773105    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773107    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773110    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773113    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773196    2576 flags.go:64] FLAG: --address="0.0.0.0"
Sep 19 10:36:30.778068 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773229    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773239    2576 flags.go:64] FLAG: --anonymous-auth="true"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773243    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773248    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773252    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773256    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773261    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773266    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773269    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773273    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773276    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773279    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773282    2576 flags.go:64] FLAG: --cgroup-root=""
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773285    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773288    2576 flags.go:64] FLAG: --client-ca-file=""
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773292    2576 flags.go:64] FLAG: --cloud-provider="external"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773294    2576 flags.go:64] FLAG: --cluster-dns="[]"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773299    2576 flags.go:64] FLAG: --cluster-domain=""
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773302    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773306    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773310    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773314    2576 flags.go:64] FLAG: --container-log-max-files="5"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773318    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 19 10:36:30.778619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773321    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773324    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773328    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773331    2576 flags.go:64] FLAG: --contention-profiling="false"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773334    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773338    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773341    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773344    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773348    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773351    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773354    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773357    2576 flags.go:64] FLAG: --enable-load-reader="false"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773360    2576 flags.go:64] FLAG: --enable-server="true"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773364    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773368    2576 flags.go:64] FLAG: --event-burst="100"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773371    2576 flags.go:64] FLAG: --event-qps="50"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773374    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773379    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773382    2576 flags.go:64] FLAG: --eviction-hard=""
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773386    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773389    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773392    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773396    2576 flags.go:64] FLAG: --eviction-soft=""
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773399    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773402    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 19 10:36:30.779181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773405    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773407    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773413    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773417    2576 flags.go:64] FLAG: --fail-swap-on="true"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773420    2576 flags.go:64] FLAG: --feature-gates=""
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773423    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773427    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773430    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773435    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773438    2576 flags.go:64] FLAG: --healthz-port="10248"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773441    2576 flags.go:64] FLAG: --help="false"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773444    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-142-165.ec2.internal"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773448    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773451    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773454    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773457    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773461    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773464    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773466    2576 flags.go:64] FLAG: --image-service-endpoint=""
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773469    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773473    2576 flags.go:64] FLAG: --kube-api-burst="100"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773476    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773479    2576 flags.go:64] FLAG: --kube-api-qps="50"
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773482    2576 flags.go:64] FLAG: --kube-reserved=""
Sep 19 10:36:30.779830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773485    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773489    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773492    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773495    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773498    2576 flags.go:64] FLAG: --lock-file=""
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773501    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773504    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773507    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773512    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773515    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773519    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773523    2576 flags.go:64] FLAG: --logging-format="text"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773526    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773529    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773532    2576 flags.go:64] FLAG: --manifest-url=""
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773536    2576 flags.go:64] FLAG: --manifest-url-header=""
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773541    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773544    2576 flags.go:64] FLAG: --max-open-files="1000000"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773548    2576 flags.go:64] FLAG: --max-pods="110"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773552    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773555    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773558    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773561    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773564    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773567    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 19 10:36:30.780870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773570    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773577    2576 flags.go:64] FLAG: --node-status-max-images="50"
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773580    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773583    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773586    2576 flags.go:64] FLAG: --pod-cidr=""
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773589    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773595    2576 flags.go:64] FLAG: --pod-manifest-path=""
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773598    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773601    2576 flags.go:64] FLAG: --pods-per-core="0"
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773605    2576 flags.go:64] FLAG: --port="10250"
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773608    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773611    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04af7bb23e7fb46b0"
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773614    2576 flags.go:64] FLAG: --qos-reserved=""
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773617    2576 flags.go:64] FLAG: --read-only-port="10255"
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773621    2576 flags.go:64] FLAG: --register-node="true"
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773624    2576 flags.go:64] FLAG: --register-with-taints=""
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773628    2576 flags.go:64] FLAG: --registry-burst="10"
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773632    2576 flags.go:64] FLAG: --registry-qps="5"
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773636    2576 flags.go:64] FLAG: --reserved-cpus=""
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773639    2576 flags.go:64] FLAG: --reserved-memory=""
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773642    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773646    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773649    2576 flags.go:64] FLAG: --rotate-certificates="false"
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773652    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773655    2576 flags.go:64] FLAG: --runonce="false"
Sep 19 10:36:30.782008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773658    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773662    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773665    2576 flags.go:64] FLAG: --seccomp-default="false"
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773668    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773671    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773674    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773677    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773681    2576 flags.go:64] FLAG: --storage-driver-password="root"
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773684    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773687    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773689    2576 flags.go:64] FLAG: --storage-driver-user="root"
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773692    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773695    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773698    2576 flags.go:64] FLAG: --system-cgroups=""
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773701    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773707    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773710    2576 flags.go:64] FLAG: --tls-cert-file=""
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773714    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773718    2576 flags.go:64] FLAG: --tls-min-version=""
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773721    2576 flags.go:64] FLAG: --tls-private-key-file=""
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773724    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773727    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773730    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773733    2576 flags.go:64] FLAG: --v="2"
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773739    2576 flags.go:64] FLAG: --version="false"
Sep 19 10:36:30.783146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773743    2576 flags.go:64] FLAG: --vmodule=""
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773747    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.773750    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773911    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773916    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773919    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773922    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773925    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773928    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773931    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773934    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773937    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773940    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773942    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773945    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773948    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773951    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773953    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773956    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773959    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 10:36:30.784181 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773962    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 10:36:30.785001 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773964    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 10:36:30.785001 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773967    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 10:36:30.785001 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773969    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 10:36:30.785001 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773973    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 10:36:30.785001 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773978    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 10:36:30.785001 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773981    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 10:36:30.785001 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773984    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 10:36:30.785001 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773987    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 10:36:30.785001 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773990    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 10:36:30.785001 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773993    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 10:36:30.785001 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.773997    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 10:36:30.785001 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774000    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 10:36:30.785001 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774003    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 10:36:30.785001 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774007    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 10:36:30.785001 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774010    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 10:36:30.785001 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774013    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 10:36:30.785001 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774016    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 10:36:30.785001 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774020    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 10:36:30.785001 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774024    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774027    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774030    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774033    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774035    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774053    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774056    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774059    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774062    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774065    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774068    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774071    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774074    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774076    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774079    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774082    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774085    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774088    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774091    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774094    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 10:36:30.785813 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774097    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774100    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774103    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774105    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774110    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774113    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774116    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774119    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774122    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774124    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774127    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774131    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774134    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774137    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774140    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774142    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774146    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774149    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774152    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774155    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 10:36:30.786453 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774158    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774160    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774163    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774166    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774169    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774171    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774174    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774176    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774179    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774182    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774185    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774187    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774191    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774193    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774196    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774199    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774203    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774205    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774208    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774211    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 10:36:30.786981 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774214    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 10:36:30.787562 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774216    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 10:36:30.787562 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.774219    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 10:36:30.787562 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.774225    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 10:36:30.787562 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.782171    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 19 10:36:30.787562 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.782196    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 19 10:36:30.787562 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.782226    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 19 10:36:30.787562 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.782236    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 19 10:36:30.787562 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782326    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 10:36:30.787562 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782335    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 10:36:30.787562 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782340    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 10:36:30.787562 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782345    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 10:36:30.787562 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782350    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 10:36:30.787562 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782354    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 10:36:30.787562 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782359    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 10:36:30.787562 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782364    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 10:36:30.787944 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782368    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 10:36:30.787944 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782372    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 10:36:30.787944 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782376    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 10:36:30.787944 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782381    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 10:36:30.787944 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782385    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 10:36:30.787944 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782390    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 10:36:30.787944 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782394    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 10:36:30.787944 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782398    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 10:36:30.787944 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782403    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 10:36:30.787944 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782408    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 10:36:30.787944 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782412    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 10:36:30.787944 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782417    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 10:36:30.787944 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782421    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 10:36:30.787944 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782425    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 10:36:30.787944 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782429    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 10:36:30.787944 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782434    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 10:36:30.787944 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782438    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 10:36:30.787944 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782442    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 10:36:30.787944 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782447    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782451    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782455    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782462    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782467    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782471    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782475    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782479    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782484    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782488    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782492    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782497    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782502    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782506    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782511    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782515    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782520    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782524    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782528    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782532    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 10:36:30.788538 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782537    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 10:36:30.789136 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782543    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 10:36:30.789136 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782550    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 10:36:30.789136 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782555    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 10:36:30.789136 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782560    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 10:36:30.789136 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782565    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 10:36:30.789136 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782569    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 10:36:30.789136 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782574    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 10:36:30.789136 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782579    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 10:36:30.789136 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782585    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 10:36:30.789136 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782592    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 10:36:30.789136 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782597    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 10:36:30.789136 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782601    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 10:36:30.789136 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782606    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 10:36:30.789136 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782610    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 10:36:30.789136 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782616    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 10:36:30.789136 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782622    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 10:36:30.789136 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782627    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 10:36:30.789136 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782631    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 10:36:30.789136 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782636    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782640    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782644    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782649    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782653    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782657    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782661    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782665    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782670    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782675    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782679    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782684    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782689    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782693    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782698    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782702    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782706    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782710    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782714    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782719    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 10:36:30.789847 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782723    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 10:36:30.790400 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782728    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 10:36:30.790400 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782734    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 10:36:30.790400 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782740    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 10:36:30.790400 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782745    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 10:36:30.790400 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782750    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 10:36:30.790400 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782755    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 10:36:30.790400 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782760    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 10:36:30.790400 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782766    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 10:36:30.790400 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782771    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 10:36:30.790400 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782776    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 10:36:30.790400 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782781    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 10:36:30.790400 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782785    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 10:36:30.790400 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782789    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 10:36:30.790400 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.782798    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 10:36:30.790400 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782986    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.782995    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783001    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783006    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783011    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783016    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783020    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783025    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783030    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783036    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783058    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783062    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783067    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783071    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783075    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783080    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783084    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783088    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783093    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783097    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 10:36:30.790890 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783101    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 10:36:30.791630 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783105    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 10:36:30.791630 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783110    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 10:36:30.791630 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783114    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 10:36:30.791630 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783121    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 10:36:30.791630 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783128    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 10:36:30.791630 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783134    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 10:36:30.791630 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783140    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 10:36:30.791630 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783147    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 10:36:30.791630 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783153    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 10:36:30.791630 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783157    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 10:36:30.791630 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783161    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 10:36:30.791630 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783165    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 10:36:30.791630 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783169    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 10:36:30.791630 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783174    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 10:36:30.791630 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783178    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 10:36:30.791630 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783183    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 10:36:30.791630 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783188    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 10:36:30.791630 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783192    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 10:36:30.791630 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783196    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 10:36:30.792121 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783201    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 10:36:30.792121 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783205    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 10:36:30.792121 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783209    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 10:36:30.792121 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783214    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 10:36:30.792121 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783218    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 10:36:30.792121 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783223    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 10:36:30.792121 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783227    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 10:36:30.792121 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783231    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 10:36:30.792121 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783236    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 10:36:30.792121 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783240    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 10:36:30.792121 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783245    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 10:36:30.792121 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783249    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 10:36:30.792121 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783253    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 10:36:30.792121 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783258    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 10:36:30.792121 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783262    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 10:36:30.792121 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783266    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 10:36:30.792121 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783271    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 10:36:30.792121 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783276    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 10:36:30.792121 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783281    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783286    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783290    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783297    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783302    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783306    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783310    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783315    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783319    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783324    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783328    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783333    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783337    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783342    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783346    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783350    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783354    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783358    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783363    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783367    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 10:36:30.792621 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783372    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783376    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783381    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783386    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783390    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783394    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783399    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783403    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783407    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783412    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783417    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783421    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783426    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783430    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783434    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783438    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783446    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783452    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783457    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783461    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 10:36:30.793201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:30.783466    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 10:36:30.793805 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.783475    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 10:36:30.793805 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.783670    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 19 10:36:30.793805 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.786301    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 19 10:36:30.793805 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.787350    2576 server.go:1019] "Starting client certificate rotation"
Sep 19 10:36:30.793805 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.787461    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 10:36:30.793805 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.787506    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 10:36:30.793805 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.790976    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 10:36:30.794798 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.794776    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 10:36:30.808417 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.808382    2576 log.go:25] "Validated CRI v1 runtime API"
Sep 19 10:36:30.810265 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.810247    2576 log.go:25] "Validated CRI v1 image API"
Sep 19 10:36:30.811826 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.811809    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 19 10:36:30.815700 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.815661    2576 fs.go:135] Filesystem UUIDs: map[268288dc-519b-4ea9-ae3c-e41a6446ede5:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 90e558ce-8dd6-4c04-882f-c669cbc1dcd2:/dev/nvme0n1p3]
Sep 19 10:36:30.815828 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.815696    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 19 10:36:30.816849 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.816821    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 10:36:30.821145 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.820979    2576 manager.go:216] Machine: {Timestamp:2026-09-19 10:36:30.819635784 +0000 UTC m=+0.315941868 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097124 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a31a20707ea4c996ae6e9986ccd08 SystemUUID:ec2a31a2-0707-ea4c-996a-e6e9986ccd08 BootID:88977e6e-4bc6-482f-b998-23ffd33e824d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ef:05:7e:73 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ef:05:7e:73 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:be:87:7b:c8:0b:18 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 19 10:36:30.821145 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.821124    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 19 10:36:30.821355 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.821315    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 19 10:36:30.824302 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.824265    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 19 10:36:30.824461 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.824307    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-165.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 19 10:36:30.825988 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.825970    2576 topology_manager.go:138] "Creating topology manager with none policy"
Sep 19 10:36:30.825988 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.825988    2576 container_manager_linux.go:309] "Creating device plugin manager"
Sep 19 10:36:30.826087 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.825997    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 10:36:30.830815 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.830795    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 10:36:30.830908 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.830898    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 19 10:36:30.834484 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.834467    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 10:36:30.834614 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.834604    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 19 10:36:30.834669 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.834660    2576 kubelet.go:486] "Attempting to sync node with API server"
Sep 19 10:36:30.834707 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.834673    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 19 10:36:30.834707 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.834689    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 19 10:36:30.834707 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.834700    2576 kubelet.go:398] "Adding apiserver pod source"
Sep 19 10:36:30.834826 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.834710    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 19 10:36:30.835107 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.835093    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 10:36:30.835171 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.835116    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 10:36:30.837154 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.837137    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 19 10:36:30.837919 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.837901    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gmtxg"
Sep 19 10:36:30.841635 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.841614    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 19 10:36:30.841754 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.841646    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 19 10:36:30.843374 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.843357    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 19 10:36:30.843374 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.843378    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 19 10:36:30.843471 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.843385    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 19 10:36:30.843471 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.843390    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 19 10:36:30.843471 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.843398    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 19 10:36:30.843471 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.843404    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 19 10:36:30.843471 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.843417    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 19 10:36:30.843471 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.843422    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 19 10:36:30.843471 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.843428    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 19 10:36:30.843471 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.843434    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 19 10:36:30.843471 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.843443    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 19 10:36:30.843471 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.843448    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 19 10:36:30.844515 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.844323    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 19 10:36:30.844515 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.844466    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 19 10:36:30.845645 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.845607    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gmtxg"
Sep 19 10:36:30.846638 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.846565    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-165.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 19 10:36:30.846638 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:30.846625    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-165.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 19 10:36:30.846766 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:30.846631    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 19 10:36:30.848913 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.848894    2576 server.go:1268] "Started kubelet"
Sep 19 10:36:30.848983 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.848955    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 19 10:36:30.849103 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.849055    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 19 10:36:30.849164 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.849132    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 19 10:36:30.849818 ip-10-0-142-165 systemd[1]: Started Kubernetes Kubelet.
Sep 19 10:36:30.850659 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.850164    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 19 10:36:30.851793 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.851777    2576 server.go:310] "Adding debug handlers to kubelet server"
Sep 19 10:36:30.854765 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:30.854730    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 19 10:36:30.857682 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.857664    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 19 10:36:30.858105 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.858083    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 19 10:36:30.858227 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:30.858208    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Sep 19 10:36:30.858691 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.858673    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 19 10:36:30.858691 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.858690    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 19 10:36:30.858810 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.858676    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 19 10:36:30.858810 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.858757    2576 reconstruct.go:97] "Volume reconstruction finished"
Sep 19 10:36:30.858810 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.858764    2576 reconciler.go:29] "Reconciler: start to sync state"
Sep 19 10:36:30.858920 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.858905    2576 factory.go:55] Registering systemd factory
Sep 19 10:36:30.858950 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.858925    2576 factory.go:223] Registration of the systemd container factory successfully
Sep 19 10:36:30.859238 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.859223    2576 factory.go:153] Registering CRI-O factory
Sep 19 10:36:30.859281 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.859243    2576 factory.go:223] Registration of the crio container factory successfully
Sep 19 10:36:30.859309 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.859300    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 19 10:36:30.859339 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.859319    2576 factory.go:103] Registering Raw factory
Sep 19 10:36:30.859339 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.859338    2576 manager.go:1196] Started watching for new ooms in manager
Sep 19 10:36:30.859804 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.859787    2576 manager.go:349] Starting recovery of all containers
Sep 19 10:36:30.860241 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.860221    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 10:36:30.862367 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:30.862338    2576 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-165.ec2.internal\" not found" node="ip-10-0-142-165.ec2.internal"
Sep 19 10:36:30.869644 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.869618    2576 manager.go:354] Recovery completed
Sep 19 10:36:30.875652 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.875635    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 10:36:30.877770 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.877748    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 10:36:30.877860 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.877784    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 10:36:30.877860 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.877795    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientPID"
Sep 19 10:36:30.878462 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.878445    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 19 10:36:30.878462 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.878457    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 10:36:30.878546 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.878463    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 19 10:36:30.878546 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.878482    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 10:36:30.879557 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.879539    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 10:36:30.879629 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.879570    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 10:36:30.879629 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.879580    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientPID"
Sep 19 10:36:30.880600 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.880586    2576 policy_none.go:49] "None policy: Start"
Sep 19 10:36:30.880639 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.880606    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 19 10:36:30.880639 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.880618    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 19 10:36:30.882573 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.882558    2576 policy_none.go:47] "Start"
Sep 19 10:36:30.919722 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.919701    2576 manager.go:337] "Starting Device Plugin manager"
Sep 19 10:36:30.942842 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:30.919745    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 19 10:36:30.942842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.919757    2576 server.go:87] "Starting device plugin registration server"
Sep 19 10:36:30.942842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.920059    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 19 10:36:30.942842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.920072    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 19 10:36:30.942842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.920169    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 19 10:36:30.942842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.920265    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 19 10:36:30.942842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.920274    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 19 10:36:30.942842 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:30.920795    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 19 10:36:30.942842 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:30.920850    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-165.ec2.internal\" not found"
Sep 19 10:36:30.992586 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.992540    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 19 10:36:30.994655 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.994628    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 19 10:36:30.994772 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.994709    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 19 10:36:30.994772 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.994745    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 19 10:36:30.994869 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:30.994797    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 19 10:36:30.996738 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:30.996711    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 10:36:31.021123 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.021005    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 10:36:31.022132 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.022112    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 10:36:31.022217 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.022150    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 10:36:31.022217 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.022161    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientPID"
Sep 19 10:36:31.022217 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.022187    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-165.ec2.internal"
Sep 19 10:36:31.033223 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.033191    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-165.ec2.internal"
Sep 19 10:36:31.033307 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:31.033226    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-165.ec2.internal\": node \"ip-10-0-142-165.ec2.internal\" not found"
Sep 19 10:36:31.047353 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:31.047312    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Sep 19 10:36:31.095739 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.095705    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal"]
Sep 19 10:36:31.095856 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.095795    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 10:36:31.096828 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.096811    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 10:36:31.096906 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.096843    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 10:36:31.096906 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.096854    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientPID"
Sep 19 10:36:31.098272 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.098259    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 10:36:31.098403 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.098389    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal"
Sep 19 10:36:31.098454 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.098422    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 10:36:31.099029 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.099012    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 10:36:31.099128 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.099014    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 10:36:31.099128 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.099067    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 10:36:31.099128 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.099078    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 10:36:31.099128 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.099080    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientPID"
Sep 19 10:36:31.099128 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.099090    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientPID"
Sep 19 10:36:31.099957 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.099944    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal"
Sep 19 10:36:31.100006 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.099972    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 10:36:31.100712 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.100694    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 10:36:31.100800 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.100721    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 10:36:31.100800 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.100731    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientPID"
Sep 19 10:36:31.124058 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:31.124009    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-165.ec2.internal\" not found" node="ip-10-0-142-165.ec2.internal"
Sep 19 10:36:31.127757 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:31.127740    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-165.ec2.internal\" not found" node="ip-10-0-142-165.ec2.internal"
Sep 19 10:36:31.148443 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:31.148415    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Sep 19 10:36:31.159682 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.159654    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5859d74372842ffe42460fcd65d85cf5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal\" (UID: \"5859d74372842ffe42460fcd65d85cf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal"
Sep 19 10:36:31.159756 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.159685    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5859d74372842ffe42460fcd65d85cf5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal\" (UID: \"5859d74372842ffe42460fcd65d85cf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal"
Sep 19 10:36:31.159756 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.159704    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a889b3717fa0a5037e893154459d9772-config\") pod \"kube-apiserver-proxy-ip-10-0-142-165.ec2.internal\" (UID: \"a889b3717fa0a5037e893154459d9772\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal"
Sep 19 10:36:31.248987 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:31.248952    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Sep 19 10:36:31.260328 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.260297    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5859d74372842ffe42460fcd65d85cf5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal\" (UID: \"5859d74372842ffe42460fcd65d85cf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal"
Sep 19 10:36:31.260393 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.260337    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5859d74372842ffe42460fcd65d85cf5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal\" (UID: \"5859d74372842ffe42460fcd65d85cf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal"
Sep 19 10:36:31.260393 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.260362    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a889b3717fa0a5037e893154459d9772-config\") pod \"kube-apiserver-proxy-ip-10-0-142-165.ec2.internal\" (UID: \"a889b3717fa0a5037e893154459d9772\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal"
Sep 19 10:36:31.260462 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.260394    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a889b3717fa0a5037e893154459d9772-config\") pod \"kube-apiserver-proxy-ip-10-0-142-165.ec2.internal\" (UID: \"a889b3717fa0a5037e893154459d9772\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal"
Sep 19 10:36:31.260462 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.260401    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5859d74372842ffe42460fcd65d85cf5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal\" (UID: \"5859d74372842ffe42460fcd65d85cf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal"
Sep 19 10:36:31.260462 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.260401    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5859d74372842ffe42460fcd65d85cf5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal\" (UID: \"5859d74372842ffe42460fcd65d85cf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal"
Sep 19 10:36:31.349798 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:31.349722    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Sep 19 10:36:31.427245 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.427198    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal"
Sep 19 10:36:31.431603 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.431311    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal"
Sep 19 10:36:31.450467 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:31.450428    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Sep 19 10:36:31.551098 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:31.551058    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Sep 19 10:36:31.651635 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:31.651573    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Sep 19 10:36:31.752268 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:31.752236    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Sep 19 10:36:31.786839 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.786812    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 19 10:36:31.787248 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.786970    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 19 10:36:31.787248 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.787007    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 19 10:36:31.850084 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.850020    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-18 10:31:30 +0000 UTC" deadline="2028-04-07 05:29:30.395008455 +0000 UTC"
Sep 19 10:36:31.850084 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:31.850075    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13578h52m58.544937909s"
Sep 19 10:36:31.853103 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:31.853068    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Sep 19 10:36:31.954304 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:31.954071    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Sep 19 10:36:32.029434 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:32.029400    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda889b3717fa0a5037e893154459d9772.slice/crio-a1f7e7a38cedc453ebfa32385c859486d20124777e5e792706757e9947ac5382 WatchSource:0}: Error finding container a1f7e7a38cedc453ebfa32385c859486d20124777e5e792706757e9947ac5382: Status 404 returned error can't find the container with id a1f7e7a38cedc453ebfa32385c859486d20124777e5e792706757e9947ac5382
Sep 19 10:36:32.029660 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:32.029640    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5859d74372842ffe42460fcd65d85cf5.slice/crio-ccd3c5e14b96607084957d0d029189e48e9a6415d3affec2a66a5225d69bd74d WatchSource:0}: Error finding container ccd3c5e14b96607084957d0d029189e48e9a6415d3affec2a66a5225d69bd74d: Status 404 returned error can't find the container with id ccd3c5e14b96607084957d0d029189e48e9a6415d3affec2a66a5225d69bd74d
Sep 19 10:36:32.034974 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.034949    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 10:36:32.055163 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:32.055129    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Sep 19 10:36:32.088617 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.088583    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 10:36:32.158342 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.158314    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal"
Sep 19 10:36:32.171393 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.171369    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 19 10:36:32.173019 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.172998    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal"
Sep 19 10:36:32.175019 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.174998    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 10:36:32.179057 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.179028    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 19 10:36:32.836183 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.835840    2576 apiserver.go:52] "Watching apiserver"
Sep 19 10:36:32.844219 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.844179    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 19 10:36:32.846220 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.846170    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-m9f9w","kube-system/konnectivity-agent-vkrwn","kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2","openshift-dns/node-resolver-5f6pp","openshift-multus/multus-additional-cni-plugins-z6nh5","openshift-multus/network-metrics-daemon-7r6w2","openshift-network-diagnostics/network-check-target-n7g7l","openshift-network-operator/iptables-alerter-s5hpc","kube-system/global-pull-secret-syncer-75sfv","openshift-cluster-node-tuning-operator/tuned-zbwz5","openshift-image-registry/node-ca-jvnt2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal","openshift-multus/multus-vscrj"]
Sep 19 10:36:32.849061 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.849015    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:32.849230 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:32.849144    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r6w2" podUID="64bed023-f7cc-46f8-b5c0-fbb2e62f56f9"
Sep 19 10:36:32.852106 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.851536    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.854594 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.853805    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.854594 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.854205    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 19 10:36:32.854594 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.854472    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 19 10:36:32.854594 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.854516    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hh5jz\""
Sep 19 10:36:32.856431 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.856407    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 19 10:36:32.856568 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.856472    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lfszl\""
Sep 19 10:36:32.856568 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.856412    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.856926 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.856900    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 19 10:36:32.857033 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.856966    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 19 10:36:32.857033 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.856904    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 19 10:36:32.858047 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.858025    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 19 10:36:32.858908 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.858891    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5f6pp"
Sep 19 10:36:32.859929 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.859908    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-f6vfh\""
Sep 19 10:36:32.860027 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.860002    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 19 10:36:32.860185 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.860170    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 19 10:36:32.860487 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.860278    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 19 10:36:32.861238 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.861217    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 19 10:36:32.863167 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.861819    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tpjx7\""
Sep 19 10:36:32.863167 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.862069    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 19 10:36:32.863167 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.862484    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.864145 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.864119    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s5hpc"
Sep 19 10:36:32.865057 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.865017    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 19 10:36:32.865504 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.865485    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 19 10:36:32.865741 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.865719    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-v4cp9\""
Sep 19 10:36:32.866764 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.866734    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:32.866885 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.866799    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 19 10:36:32.866885 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:32.866823    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7g7l" podUID="b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4"
Sep 19 10:36:32.867031 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.866923    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 19 10:36:32.867031 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.866947    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 19 10:36:32.867357 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.867337    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hbq2z\""
Sep 19 10:36:32.868205 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.868184    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-etc-kubernetes\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.868871 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.868215    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-lib-modules\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.868871 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.868242    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-var-lib-kubelet\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.868871 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.868269    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e10a7fca-fab2-40fe-888c-425f875e9a3e-system-cni-dir\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.868871 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.868299    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e10a7fca-fab2-40fe-888c-425f875e9a3e-cnibin\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.868871 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.868330    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e10a7fca-fab2-40fe-888c-425f875e9a3e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.868871 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.868360    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e10a7fca-fab2-40fe-888c-425f875e9a3e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.868871 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.868386    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-tuned\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.868871 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.868430    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/96999fca-94e0-4248-b251-eb47595cfd07-registration-dir\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.868871 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.868456    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/96999fca-94e0-4248-b251-eb47595cfd07-etc-selinux\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.868871 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.868527    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d-tmp-dir\") pod \"node-resolver-5f6pp\" (UID: \"e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d\") " pod="openshift-dns/node-resolver-5f6pp"
Sep 19 10:36:32.868871 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.868601    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt2gz\" (UniqueName: \"kubernetes.io/projected/e10a7fca-fab2-40fe-888c-425f875e9a3e-kube-api-access-xt2gz\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.868871 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.868647    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-host\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.868871 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.868684    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-host-var-lib-cni-bin\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.868871 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.868710    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a3b17f10-243c-4741-ad8d-feb11c375cbb-multus-daemon-config\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.869770 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.868736    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/96999fca-94e0-4248-b251-eb47595cfd07-sys-fs\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.869770 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.869206    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dmpw\" (UniqueName: \"kubernetes.io/projected/96999fca-94e0-4248-b251-eb47595cfd07-kube-api-access-8dmpw\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.869770 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.869278    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d-hosts-file\") pod \"node-resolver-5f6pp\" (UID: \"e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d\") " pod="openshift-dns/node-resolver-5f6pp"
Sep 19 10:36:32.869770 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.869319    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcpjp\" (UniqueName: \"kubernetes.io/projected/e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d-kube-api-access-lcpjp\") pod \"node-resolver-5f6pp\" (UID: \"e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d\") " pod="openshift-dns/node-resolver-5f6pp"
Sep 19 10:36:32.869770 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.869378    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e10a7fca-fab2-40fe-888c-425f875e9a3e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.869770 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.869428    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-tmp\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.871137 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.871074    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-cnibin\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.871265 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.871154    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-host-run-netns\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.871265 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.871195    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-hostroot\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.871265 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.871234    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-multus-conf-dir\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.871441 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.871274    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-host-run-multus-certs\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.871441 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.871313    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-sysctl-d\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.871562 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.871460    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-systemd\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.871562 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.871494    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-sys\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.871562 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.871549    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-host-var-lib-cni-multus\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.871740 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.871616    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/96999fca-94e0-4248-b251-eb47595cfd07-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.871740 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.871654    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/96999fca-94e0-4248-b251-eb47595cfd07-socket-dir\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.871740 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.871669    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vkrwn"
Sep 19 10:36:32.871740 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.871691    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/96999fca-94e0-4248-b251-eb47595cfd07-device-dir\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.871985 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.871737    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvjqm\" (UniqueName: \"kubernetes.io/projected/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-kube-api-access-mvjqm\") pod \"network-metrics-daemon-7r6w2\" (UID: \"64bed023-f7cc-46f8-b5c0-fbb2e62f56f9\") " pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:32.871985 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.871798    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-sysconfig\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.871985 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.871849    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-multus-cni-dir\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.871985 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.871887    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e10a7fca-fab2-40fe-888c-425f875e9a3e-cni-binary-copy\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.871985 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.871956    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-modprobe-d\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.872293 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.871997    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-kubernetes\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.872293 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.872036    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-sysctl-conf\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.872293 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.872076    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-host-var-lib-kubelet\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.872293 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.872245    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h559c\" (UniqueName: \"kubernetes.io/projected/a3b17f10-243c-4741-ad8d-feb11c375cbb-kube-api-access-h559c\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.872519 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.872478    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs\") pod \"network-metrics-daemon-7r6w2\" (UID: \"64bed023-f7cc-46f8-b5c0-fbb2e62f56f9\") " pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:32.872587 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.872525    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-run\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.872652 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.872598    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jb69\" (UniqueName: \"kubernetes.io/projected/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-kube-api-access-8jb69\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.872652 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.872644    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-system-cni-dir\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.872777 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.872713    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3b17f10-243c-4741-ad8d-feb11c375cbb-cni-binary-copy\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.872777 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.872753    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-multus-socket-dir-parent\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.872900 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.872819    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 10:36:32.872900 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.872821    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-host-run-k8s-cni-cncf-io\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.874741 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.874166    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 19 10:36:32.874741 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.874250    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:32.874741 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:32.874385    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75sfv" podUID="acbcb3f6-b65e-4ee4-b497-e5c913e646d7"
Sep 19 10:36:32.875773 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.875644    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qm8ss\""
Sep 19 10:36:32.875968 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.875952    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 19 10:36:32.878182 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.878150    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 10:36:32.878320 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.878162    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 10:36:32.878546 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.878524    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.881197 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.881172    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jvnt2"
Sep 19 10:36:32.881410 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.881205    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ln4ff\""
Sep 19 10:36:32.881410 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.881330    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 19 10:36:32.881532 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.881411    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 19 10:36:32.881681 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.881663    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 19 10:36:32.881771 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.881752    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 19 10:36:32.881838 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.881764    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 19 10:36:32.882485 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.882312    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 19 10:36:32.884648 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.884152    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 19 10:36:32.884648 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.884508    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-26cw7\""
Sep 19 10:36:32.884897 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.884879    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 19 10:36:32.885017 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.884998    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 19 10:36:32.894685 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.894644    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cvtzj"
Sep 19 10:36:32.904869 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.904838    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cvtzj"
Sep 19 10:36:32.959726 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.959697    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 19 10:36:32.973046 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e10a7fca-fab2-40fe-888c-425f875e9a3e-cni-binary-copy\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.973046 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973052    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-dbus\") pod \"global-pull-secret-syncer-75sfv\" (UID: \"acbcb3f6-b65e-4ee4-b497-e5c913e646d7\") " pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:32.973306 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973075    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-etc-openvswitch\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.973306 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973099    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-run-openvswitch\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.973306 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973142    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-node-log\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.973306 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973165    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-log-socket\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.973306 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973211    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-modprobe-d\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.973306 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973274    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-sysctl-conf\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.973559 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973391    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-modprobe-d\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.973559 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973384    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-host-var-lib-kubelet\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.973559 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973448    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-run-systemd\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.973559 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973481    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.973559 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973514    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5ssg\" (UniqueName: \"kubernetes.io/projected/eafc0cde-0b81-4e6e-83b9-3967ed105371-kube-api-access-c5ssg\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.973559 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973517    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-host-var-lib-kubelet\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.973559 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973448    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-sysctl-conf\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.973559 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973540    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsnxf\" (UniqueName: \"kubernetes.io/projected/a48975ea-c138-4925-9447-74c6df9a3c36-kube-api-access-jsnxf\") pod \"iptables-alerter-s5hpc\" (UID: \"a48975ea-c138-4925-9447-74c6df9a3c36\") " pod="openshift-network-operator/iptables-alerter-s5hpc"
Sep 19 10:36:32.974047 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973593    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs\") pod \"network-metrics-daemon-7r6w2\" (UID: \"64bed023-f7cc-46f8-b5c0-fbb2e62f56f9\") " pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:32.974047 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973676    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-run\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.974047 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973719    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jb69\" (UniqueName: \"kubernetes.io/projected/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-kube-api-access-8jb69\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.974047 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973745    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-multus-socket-dir-parent\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.974047 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:32.973782    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:36:32.974047 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973835    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-etc-kubernetes\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.974047 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973847    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e10a7fca-fab2-40fe-888c-425f875e9a3e-cni-binary-copy\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.974047 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973884    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-run\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.974047 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973978    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-multus-socket-dir-parent\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.974047 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.973786    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-etc-kubernetes\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.974455 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:32.974080    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs podName:64bed023-f7cc-46f8-b5c0-fbb2e62f56f9 nodeName:}" failed. No retries permitted until 2026-09-19 10:36:33.474051246 +0000 UTC m=+2.970357345 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs") pod "network-metrics-daemon-7r6w2" (UID: "64bed023-f7cc-46f8-b5c0-fbb2e62f56f9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:36:32.974455 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974074    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e10a7fca-fab2-40fe-888c-425f875e9a3e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.974455 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974149    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-run-netns\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.974455 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974182    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmgsj\" (UniqueName: \"kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj\") pod \"network-check-target-n7g7l\" (UID: \"b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4\") " pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:32.974455 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974205    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e10a7fca-fab2-40fe-888c-425f875e9a3e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.974455 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974217    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-lib-modules\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.974455 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974247    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e10a7fca-fab2-40fe-888c-425f875e9a3e-system-cni-dir\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.974455 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974273    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-cni-netd\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.974455 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974306    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e10a7fca-fab2-40fe-888c-425f875e9a3e-system-cni-dir\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.974455 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974314    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3bda56d0-e1c8-42c2-8905-9719bd0eace8-serviceca\") pod \"node-ca-jvnt2\" (UID: \"3bda56d0-e1c8-42c2-8905-9719bd0eace8\") " pod="openshift-image-registry/node-ca-jvnt2"
Sep 19 10:36:32.974455 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974362    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-tuned\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.974455 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974381    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d-tmp-dir\") pod \"node-resolver-5f6pp\" (UID: \"e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d\") " pod="openshift-dns/node-resolver-5f6pp"
Sep 19 10:36:32.974455 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974399    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eafc0cde-0b81-4e6e-83b9-3967ed105371-ovnkube-script-lib\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.974455 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974361    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-lib-modules\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.974455 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974436    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-host\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.975225 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974502    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-host-var-lib-cni-bin\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.975225 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974507    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-host\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.975225 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974560    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a3b17f10-243c-4741-ad8d-feb11c375cbb-multus-daemon-config\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.975225 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974588    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d-hosts-file\") pod \"node-resolver-5f6pp\" (UID: \"e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d\") " pod="openshift-dns/node-resolver-5f6pp"
Sep 19 10:36:32.975225 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974633    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d-hosts-file\") pod \"node-resolver-5f6pp\" (UID: \"e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d\") " pod="openshift-dns/node-resolver-5f6pp"
Sep 19 10:36:32.975225 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974659    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-host-var-lib-cni-bin\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.975225 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974713    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 19 10:36:32.975225 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.974742    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d-tmp-dir\") pod \"node-resolver-5f6pp\" (UID: \"e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d\") " pod="openshift-dns/node-resolver-5f6pp"
Sep 19 10:36:32.975625 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.975273    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1ec6f61d-3386-407c-8be3-9ba0dfe3045d-konnectivity-ca\") pod \"konnectivity-agent-vkrwn\" (UID: \"1ec6f61d-3386-407c-8be3-9ba0dfe3045d\") " pod="kube-system/konnectivity-agent-vkrwn"
Sep 19 10:36:32.975625 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.975316    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-tmp\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.975625 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.975349    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-host-run-netns\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.975625 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.975377    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-hostroot\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.975625 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.975402    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-multus-conf-dir\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.975625 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.975431    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-cni-bin\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.975625 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.975457    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eafc0cde-0b81-4e6e-83b9-3967ed105371-ovnkube-config\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.975625 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.975484    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-systemd\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.975625 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.975511    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/96999fca-94e0-4248-b251-eb47595cfd07-socket-dir\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.975625 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.975535    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3bda56d0-e1c8-42c2-8905-9719bd0eace8-host\") pod \"node-ca-jvnt2\" (UID: \"3bda56d0-e1c8-42c2-8905-9719bd0eace8\") " pod="openshift-image-registry/node-ca-jvnt2"
Sep 19 10:36:32.975625 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.975568    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvjqm\" (UniqueName: \"kubernetes.io/projected/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-kube-api-access-mvjqm\") pod \"network-metrics-daemon-7r6w2\" (UID: \"64bed023-f7cc-46f8-b5c0-fbb2e62f56f9\") " pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:32.975625 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.975595    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-sysconfig\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.975625 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.975608    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a3b17f10-243c-4741-ad8d-feb11c375cbb-multus-daemon-config\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.976335 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.975696    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-host-run-netns\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.976335 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.975744    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-hostroot\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.976335 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.975789    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-multus-conf-dir\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.976335 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.975862    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-systemd\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.976335 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.975980    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/96999fca-94e0-4248-b251-eb47595cfd07-socket-dir\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.976335 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976112    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-sysconfig\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.976335 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.975623    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-run-ovn-kubernetes\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.976335 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976199    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-kubernetes\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.976335 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976249    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h559c\" (UniqueName: \"kubernetes.io/projected/a3b17f10-243c-4741-ad8d-feb11c375cbb-kube-api-access-h559c\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.976335 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976283    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret\") pod \"global-pull-secret-syncer-75sfv\" (UID: \"acbcb3f6-b65e-4ee4-b497-e5c913e646d7\") " pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:32.976335 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976311    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-slash\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.976335 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-system-cni-dir\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.976941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976352    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-kubernetes\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.976941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976362    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3b17f10-243c-4741-ad8d-feb11c375cbb-cni-binary-copy\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.976941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976412    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-host-run-k8s-cni-cncf-io\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.976941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976444    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e10a7fca-fab2-40fe-888c-425f875e9a3e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.976941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976474    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1ec6f61d-3386-407c-8be3-9ba0dfe3045d-agent-certs\") pod \"konnectivity-agent-vkrwn\" (UID: \"1ec6f61d-3386-407c-8be3-9ba0dfe3045d\") " pod="kube-system/konnectivity-agent-vkrwn"
Sep 19 10:36:32.976941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976502    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-kubelet-config\") pod \"global-pull-secret-syncer-75sfv\" (UID: \"acbcb3f6-b65e-4ee4-b497-e5c913e646d7\") " pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:32.976941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976527    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-var-lib-openvswitch\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.976941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976555    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a48975ea-c138-4925-9447-74c6df9a3c36-iptables-alerter-script\") pod \"iptables-alerter-s5hpc\" (UID: \"a48975ea-c138-4925-9447-74c6df9a3c36\") " pod="openshift-network-operator/iptables-alerter-s5hpc"
Sep 19 10:36:32.976941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976583    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-var-lib-kubelet\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.976941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976608    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e10a7fca-fab2-40fe-888c-425f875e9a3e-cnibin\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.976941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976634    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-systemd-units\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.976941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976688    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a48975ea-c138-4925-9447-74c6df9a3c36-host-slash\") pod \"iptables-alerter-s5hpc\" (UID: \"a48975ea-c138-4925-9447-74c6df9a3c36\") " pod="openshift-network-operator/iptables-alerter-s5hpc"
Sep 19 10:36:32.976941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976716    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/96999fca-94e0-4248-b251-eb47595cfd07-registration-dir\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.976941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976741    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/96999fca-94e0-4248-b251-eb47595cfd07-etc-selinux\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.976941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976768    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt2gz\" (UniqueName: \"kubernetes.io/projected/e10a7fca-fab2-40fe-888c-425f875e9a3e-kube-api-access-xt2gz\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.976941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976793    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/96999fca-94e0-4248-b251-eb47595cfd07-sys-fs\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.976941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976793    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-host-run-k8s-cni-cncf-io\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.977797 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976838    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dmpw\" (UniqueName: \"kubernetes.io/projected/96999fca-94e0-4248-b251-eb47595cfd07-kube-api-access-8dmpw\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.977797 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976870    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcpjp\" (UniqueName: \"kubernetes.io/projected/e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d-kube-api-access-lcpjp\") pod \"node-resolver-5f6pp\" (UID: \"e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d\") " pod="openshift-dns/node-resolver-5f6pp"
Sep 19 10:36:32.977797 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976899    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e10a7fca-fab2-40fe-888c-425f875e9a3e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.977797 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976930    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-kubelet\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.977797 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976957    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eafc0cde-0b81-4e6e-83b9-3967ed105371-ovn-node-metrics-cert\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.977797 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.976986    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-cnibin\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.977797 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-host-run-multus-certs\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.977797 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977042    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eafc0cde-0b81-4e6e-83b9-3967ed105371-env-overrides\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.977797 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977057    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3b17f10-243c-4741-ad8d-feb11c375cbb-cni-binary-copy\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.977797 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977071    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-sysctl-d\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.977797 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977151    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-sys\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.977797 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977164    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-system-cni-dir\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.977797 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977180    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-host-var-lib-cni-multus\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.977797 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977208    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/96999fca-94e0-4248-b251-eb47595cfd07-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.977797 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977242    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/96999fca-94e0-4248-b251-eb47595cfd07-sys-fs\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.977797 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977320    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-var-lib-kubelet\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.977797 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977346    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e10a7fca-fab2-40fe-888c-425f875e9a3e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.978711 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977360    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e10a7fca-fab2-40fe-888c-425f875e9a3e-cnibin\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.978711 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977426    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/96999fca-94e0-4248-b251-eb47595cfd07-registration-dir\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.978711 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977457    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/96999fca-94e0-4248-b251-eb47595cfd07-etc-selinux\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.978711 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977490    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/96999fca-94e0-4248-b251-eb47595cfd07-device-dir\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.978711 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977530    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-run-ovn\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:32.978711 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977561    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6srgc\" (UniqueName: \"kubernetes.io/projected/3bda56d0-e1c8-42c2-8905-9719bd0eace8-kube-api-access-6srgc\") pod \"node-ca-jvnt2\" (UID: \"3bda56d0-e1c8-42c2-8905-9719bd0eace8\") " pod="openshift-image-registry/node-ca-jvnt2"
Sep 19 10:36:32.978711 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977627    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/96999fca-94e0-4248-b251-eb47595cfd07-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.978711 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977668    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-cnibin\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.978711 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977681    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-host-var-lib-cni-multus\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.978711 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977749    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-host-run-multus-certs\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.978711 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977844    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-sysctl-d\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.978711 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977890    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-sys\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.978711 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977909    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-multus-cni-dir\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.978711 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977921    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e10a7fca-fab2-40fe-888c-425f875e9a3e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.978711 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.977961    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/96999fca-94e0-4248-b251-eb47595cfd07-device-dir\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.978711 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.978169    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3b17f10-243c-4741-ad8d-feb11c375cbb-multus-cni-dir\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:32.987467 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.987309    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-etc-tuned\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.987467 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.987329    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-tmp\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.994636 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.994583    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dmpw\" (UniqueName: \"kubernetes.io/projected/96999fca-94e0-4248-b251-eb47595cfd07-kube-api-access-8dmpw\") pod \"aws-ebs-csi-driver-node-mkcm2\" (UID: \"96999fca-94e0-4248-b251-eb47595cfd07\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:32.994636 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.994577    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvjqm\" (UniqueName: \"kubernetes.io/projected/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-kube-api-access-mvjqm\") pod \"network-metrics-daemon-7r6w2\" (UID: \"64bed023-f7cc-46f8-b5c0-fbb2e62f56f9\") " pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:32.994845 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.994678    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt2gz\" (UniqueName: \"kubernetes.io/projected/e10a7fca-fab2-40fe-888c-425f875e9a3e-kube-api-access-xt2gz\") pod \"multus-additional-cni-plugins-z6nh5\" (UID: \"e10a7fca-fab2-40fe-888c-425f875e9a3e\") " pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:32.995350 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.995315    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcpjp\" (UniqueName: \"kubernetes.io/projected/e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d-kube-api-access-lcpjp\") pod \"node-resolver-5f6pp\" (UID: \"e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d\") " pod="openshift-dns/node-resolver-5f6pp"
Sep 19 10:36:32.995904 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.995879    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jb69\" (UniqueName: \"kubernetes.io/projected/5f7d5cb0-8c94-4757-967a-dbd6c59fc53d-kube-api-access-8jb69\") pod \"tuned-zbwz5\" (UID: \"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d\") " pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:32.996234 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:32.996150    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h559c\" (UniqueName: \"kubernetes.io/projected/a3b17f10-243c-4741-ad8d-feb11c375cbb-kube-api-access-h559c\") pod \"multus-vscrj\" (UID: \"a3b17f10-243c-4741-ad8d-feb11c375cbb\") " pod="openshift-multus/multus-vscrj"
Sep 19 10:36:33.001388 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.001018    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal" event={"ID":"a889b3717fa0a5037e893154459d9772","Type":"ContainerStarted","Data":"a1f7e7a38cedc453ebfa32385c859486d20124777e5e792706757e9947ac5382"}
Sep 19 10:36:33.002578 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.002548    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal" event={"ID":"5859d74372842ffe42460fcd65d85cf5","Type":"ContainerStarted","Data":"ccd3c5e14b96607084957d0d029189e48e9a6415d3affec2a66a5225d69bd74d"}
Sep 19 10:36:33.079097 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079018    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-kubelet\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.079097 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079103    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eafc0cde-0b81-4e6e-83b9-3967ed105371-ovn-node-metrics-cert\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.079378 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079138    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eafc0cde-0b81-4e6e-83b9-3967ed105371-env-overrides\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.079378 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079199    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-kubelet\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.079378 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079271    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-run-ovn\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.079378 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079304    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6srgc\" (UniqueName: \"kubernetes.io/projected/3bda56d0-e1c8-42c2-8905-9719bd0eace8-kube-api-access-6srgc\") pod \"node-ca-jvnt2\" (UID: \"3bda56d0-e1c8-42c2-8905-9719bd0eace8\") " pod="openshift-image-registry/node-ca-jvnt2"
Sep 19 10:36:33.079378 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079338    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-dbus\") pod \"global-pull-secret-syncer-75sfv\" (UID: \"acbcb3f6-b65e-4ee4-b497-e5c913e646d7\") " pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:33.079378 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079364    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-etc-openvswitch\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.079837 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079407    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-run-ovn\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.079837 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079419    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-run-openvswitch\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.079837 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079475    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-run-openvswitch\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.079837 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079474    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-node-log\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.079837 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079511    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-node-log\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.079837 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079514    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-log-socket\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.079837 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079548    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-log-socket\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.079837 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079553    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-run-systemd\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.079837 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079414    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-etc-openvswitch\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.079837 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079584    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.079837 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079628    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5ssg\" (UniqueName: \"kubernetes.io/projected/eafc0cde-0b81-4e6e-83b9-3967ed105371-kube-api-access-c5ssg\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.079837 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079628    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.079837 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079642    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-run-systemd\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.079837 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079666    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsnxf\" (UniqueName: \"kubernetes.io/projected/a48975ea-c138-4925-9447-74c6df9a3c36-kube-api-access-jsnxf\") pod \"iptables-alerter-s5hpc\" (UID: \"a48975ea-c138-4925-9447-74c6df9a3c36\") " pod="openshift-network-operator/iptables-alerter-s5hpc"
Sep 19 10:36:33.079837 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079702    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-dbus\") pod \"global-pull-secret-syncer-75sfv\" (UID: \"acbcb3f6-b65e-4ee4-b497-e5c913e646d7\") " pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:33.079837 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079767    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eafc0cde-0b81-4e6e-83b9-3967ed105371-env-overrides\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.080522 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079914    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-run-netns\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.080522 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079946    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgsj\" (UniqueName: \"kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj\") pod \"network-check-target-n7g7l\" (UID: \"b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4\") " pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:33.080522 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.079979    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-cni-netd\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.080522 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.080004    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3bda56d0-e1c8-42c2-8905-9719bd0eace8-serviceca\") pod \"node-ca-jvnt2\" (UID: \"3bda56d0-e1c8-42c2-8905-9719bd0eace8\") " pod="openshift-image-registry/node-ca-jvnt2"
Sep 19 10:36:33.080522 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.080033    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eafc0cde-0b81-4e6e-83b9-3967ed105371-ovnkube-script-lib\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.080522 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.080069    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-run-netns\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.080522 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.080084    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1ec6f61d-3386-407c-8be3-9ba0dfe3045d-konnectivity-ca\") pod \"konnectivity-agent-vkrwn\" (UID: \"1ec6f61d-3386-407c-8be3-9ba0dfe3045d\") " pod="kube-system/konnectivity-agent-vkrwn"
Sep 19 10:36:33.080522 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.080459    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-cni-netd\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.080522 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.080516    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-cni-bin\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.080886 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.080549    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eafc0cde-0b81-4e6e-83b9-3967ed105371-ovnkube-config\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.080886 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.080578    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3bda56d0-e1c8-42c2-8905-9719bd0eace8-host\") pod \"node-ca-jvnt2\" (UID: \"3bda56d0-e1c8-42c2-8905-9719bd0eace8\") " pod="openshift-image-registry/node-ca-jvnt2"
Sep 19 10:36:33.080886 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.080608    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-run-ovn-kubernetes\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.080886 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.080613    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3bda56d0-e1c8-42c2-8905-9719bd0eace8-serviceca\") pod \"node-ca-jvnt2\" (UID: \"3bda56d0-e1c8-42c2-8905-9719bd0eace8\") " pod="openshift-image-registry/node-ca-jvnt2"
Sep 19 10:36:33.080886 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.080640    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret\") pod \"global-pull-secret-syncer-75sfv\" (UID: \"acbcb3f6-b65e-4ee4-b497-e5c913e646d7\") " pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:33.080886 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.080766    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1ec6f61d-3386-407c-8be3-9ba0dfe3045d-konnectivity-ca\") pod \"konnectivity-agent-vkrwn\" (UID: \"1ec6f61d-3386-407c-8be3-9ba0dfe3045d\") " pod="kube-system/konnectivity-agent-vkrwn"
Sep 19 10:36:33.080886 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.080824    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-slash\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.080886 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.080835    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-run-ovn-kubernetes\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.080886 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.080838    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3bda56d0-e1c8-42c2-8905-9719bd0eace8-host\") pod \"node-ca-jvnt2\" (UID: \"3bda56d0-e1c8-42c2-8905-9719bd0eace8\") " pod="openshift-image-registry/node-ca-jvnt2"
Sep 19 10:36:33.080886 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.080880    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-slash\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.080886 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.080890    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-host-cni-bin\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.081403 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:33.080932    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 10:36:33.081403 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.080969    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1ec6f61d-3386-407c-8be3-9ba0dfe3045d-agent-certs\") pod \"konnectivity-agent-vkrwn\" (UID: \"1ec6f61d-3386-407c-8be3-9ba0dfe3045d\") " pod="kube-system/konnectivity-agent-vkrwn"
Sep 19 10:36:33.081403 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:33.081004    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret podName:acbcb3f6-b65e-4ee4-b497-e5c913e646d7 nodeName:}" failed. No retries permitted until 2026-09-19 10:36:33.580983004 +0000 UTC m=+3.077289081 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret") pod "global-pull-secret-syncer-75sfv" (UID: "acbcb3f6-b65e-4ee4-b497-e5c913e646d7") : object "kube-system"/"original-pull-secret" not registered
Sep 19 10:36:33.081403 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.081039    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-kubelet-config\") pod \"global-pull-secret-syncer-75sfv\" (UID: \"acbcb3f6-b65e-4ee4-b497-e5c913e646d7\") " pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:33.081403 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.081137    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-var-lib-openvswitch\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.081403 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.081147    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eafc0cde-0b81-4e6e-83b9-3967ed105371-ovnkube-script-lib\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.081403 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.081192    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a48975ea-c138-4925-9447-74c6df9a3c36-iptables-alerter-script\") pod \"iptables-alerter-s5hpc\" (UID: \"a48975ea-c138-4925-9447-74c6df9a3c36\") " pod="openshift-network-operator/iptables-alerter-s5hpc"
Sep 19 10:36:33.081403 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.081222    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-kubelet-config\") pod \"global-pull-secret-syncer-75sfv\" (UID: \"acbcb3f6-b65e-4ee4-b497-e5c913e646d7\") " pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:33.081403 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.081266    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-var-lib-openvswitch\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.081403 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.081229    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-systemd-units\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.081403 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.081323    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a48975ea-c138-4925-9447-74c6df9a3c36-host-slash\") pod \"iptables-alerter-s5hpc\" (UID: \"a48975ea-c138-4925-9447-74c6df9a3c36\") " pod="openshift-network-operator/iptables-alerter-s5hpc"
Sep 19 10:36:33.081403 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.081326    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eafc0cde-0b81-4e6e-83b9-3967ed105371-systemd-units\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.081955 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.081438    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a48975ea-c138-4925-9447-74c6df9a3c36-host-slash\") pod \"iptables-alerter-s5hpc\" (UID: \"a48975ea-c138-4925-9447-74c6df9a3c36\") " pod="openshift-network-operator/iptables-alerter-s5hpc"
Sep 19 10:36:33.081955 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.081453    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eafc0cde-0b81-4e6e-83b9-3967ed105371-ovnkube-config\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.081955 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.081746    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a48975ea-c138-4925-9447-74c6df9a3c36-iptables-alerter-script\") pod \"iptables-alerter-s5hpc\" (UID: \"a48975ea-c138-4925-9447-74c6df9a3c36\") " pod="openshift-network-operator/iptables-alerter-s5hpc"
Sep 19 10:36:33.083119 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.083090    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eafc0cde-0b81-4e6e-83b9-3967ed105371-ovn-node-metrics-cert\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.084340 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.084316    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1ec6f61d-3386-407c-8be3-9ba0dfe3045d-agent-certs\") pod \"konnectivity-agent-vkrwn\" (UID: \"1ec6f61d-3386-407c-8be3-9ba0dfe3045d\") " pod="kube-system/konnectivity-agent-vkrwn"
Sep 19 10:36:33.089265 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:33.089164    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 10:36:33.089265 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:33.089201    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 10:36:33.089265 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:33.089218    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wmgsj for pod openshift-network-diagnostics/network-check-target-n7g7l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:36:33.089557 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:33.089300    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj podName:b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4 nodeName:}" failed. No retries permitted until 2026-09-19 10:36:33.589278265 +0000 UTC m=+3.085584366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wmgsj" (UniqueName: "kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj") pod "network-check-target-n7g7l" (UID: "b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:36:33.091020 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.090988    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6srgc\" (UniqueName: \"kubernetes.io/projected/3bda56d0-e1c8-42c2-8905-9719bd0eace8-kube-api-access-6srgc\") pod \"node-ca-jvnt2\" (UID: \"3bda56d0-e1c8-42c2-8905-9719bd0eace8\") " pod="openshift-image-registry/node-ca-jvnt2"
Sep 19 10:36:33.092246 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.092222    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsnxf\" (UniqueName: \"kubernetes.io/projected/a48975ea-c138-4925-9447-74c6df9a3c36-kube-api-access-jsnxf\") pod \"iptables-alerter-s5hpc\" (UID: \"a48975ea-c138-4925-9447-74c6df9a3c36\") " pod="openshift-network-operator/iptables-alerter-s5hpc"
Sep 19 10:36:33.092723 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.092667    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5ssg\" (UniqueName: \"kubernetes.io/projected/eafc0cde-0b81-4e6e-83b9-3967ed105371-kube-api-access-c5ssg\") pod \"ovnkube-node-m9f9w\" (UID: \"eafc0cde-0b81-4e6e-83b9-3967ed105371\") " pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.177778 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.177743    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zbwz5"
Sep 19 10:36:33.189198 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.189161    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vscrj"
Sep 19 10:36:33.198420 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.198364    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2"
Sep 19 10:36:33.205754 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.205698    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5f6pp"
Sep 19 10:36:33.215980 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.215946    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z6nh5"
Sep 19 10:36:33.226256 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.226224    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s5hpc"
Sep 19 10:36:33.235336 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.235298    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vkrwn"
Sep 19 10:36:33.244557 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.244518    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:36:33.253950 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.253917    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jvnt2"
Sep 19 10:36:33.484211 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.484119    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs\") pod \"network-metrics-daemon-7r6w2\" (UID: \"64bed023-f7cc-46f8-b5c0-fbb2e62f56f9\") " pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:33.484359 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:33.484257    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:36:33.484359 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:33.484330    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs podName:64bed023-f7cc-46f8-b5c0-fbb2e62f56f9 nodeName:}" failed. No retries permitted until 2026-09-19 10:36:34.484313976 +0000 UTC m=+3.980620047 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs") pod "network-metrics-daemon-7r6w2" (UID: "64bed023-f7cc-46f8-b5c0-fbb2e62f56f9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:36:33.585353 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.585313    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret\") pod \"global-pull-secret-syncer-75sfv\" (UID: \"acbcb3f6-b65e-4ee4-b497-e5c913e646d7\") " pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:33.585557 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:33.585452    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 10:36:33.585557 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:33.585530    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret podName:acbcb3f6-b65e-4ee4-b497-e5c913e646d7 nodeName:}" failed. No retries permitted until 2026-09-19 10:36:34.585509861 +0000 UTC m=+4.081815955 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret") pod "global-pull-secret-syncer-75sfv" (UID: "acbcb3f6-b65e-4ee4-b497-e5c913e646d7") : object "kube-system"/"original-pull-secret" not registered
Sep 19 10:36:33.686451 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.686404    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgsj\" (UniqueName: \"kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj\") pod \"network-check-target-n7g7l\" (UID: \"b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4\") " pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:33.686670 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:33.686601    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 10:36:33.686670 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:33.686628    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 10:36:33.686670 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:33.686641    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wmgsj for pod openshift-network-diagnostics/network-check-target-n7g7l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:36:33.686846 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:33.686708    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj podName:b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4 nodeName:}" failed. No retries permitted until 2026-09-19 10:36:34.686691746 +0000 UTC m=+4.182997838 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wmgsj" (UniqueName: "kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj") pod "network-check-target-n7g7l" (UID: "b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:36:33.908808 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.908695    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 10:31:32 +0000 UTC" deadline="2028-07-04 01:10:41.485484244 +0000 UTC"
Sep 19 10:36:33.908808 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:33.908747    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15686h34m7.57674319s"
Sep 19 10:36:34.181825 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:34.181638    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96999fca_94e0_4248_b251_eb47595cfd07.slice/crio-7c4c11ed41dbc490ee29218c288c6a0a62e0244d365b72162eef975979fd4066 WatchSource:0}: Error finding container 7c4c11ed41dbc490ee29218c288c6a0a62e0244d365b72162eef975979fd4066: Status 404 returned error can't find the container with id 7c4c11ed41dbc490ee29218c288c6a0a62e0244d365b72162eef975979fd4066
Sep 19 10:36:34.184367 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:34.184114    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode10a7fca_fab2_40fe_888c_425f875e9a3e.slice/crio-466a659b4859669667a2012022621d5d10928e0b2c7ba9f24db8e6952619ced5 WatchSource:0}: Error finding container 466a659b4859669667a2012022621d5d10928e0b2c7ba9f24db8e6952619ced5: Status 404 returned error can't find the container with id 466a659b4859669667a2012022621d5d10928e0b2c7ba9f24db8e6952619ced5
Sep 19 10:36:34.184821 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:34.184794    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ec6f61d_3386_407c_8be3_9ba0dfe3045d.slice/crio-574b4881dba2623e45eae567c4a76ac9d52068219c8879e465dae69122ae2c8e WatchSource:0}: Error finding container 574b4881dba2623e45eae567c4a76ac9d52068219c8879e465dae69122ae2c8e: Status 404 returned error can't find the container with id 574b4881dba2623e45eae567c4a76ac9d52068219c8879e465dae69122ae2c8e
Sep 19 10:36:34.187116 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:36:34.186945    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda48975ea_c138_4925_9447_74c6df9a3c36.slice/crio-021ec1711630a9e80af1434c3a0e4cfcf9254a0f758346eacadeca664b4236af WatchSource:0}: Error finding container 021ec1711630a9e80af1434c3a0e4cfcf9254a0f758346eacadeca664b4236af: Status 404 returned error can't find the container with id 021ec1711630a9e80af1434c3a0e4cfcf9254a0f758346eacadeca664b4236af
Sep 19 10:36:34.494429 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:34.494377    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs\") pod \"network-metrics-daemon-7r6w2\" (UID: \"64bed023-f7cc-46f8-b5c0-fbb2e62f56f9\") " pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:34.494599 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:34.494544    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:36:34.494644 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:34.494613    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs podName:64bed023-f7cc-46f8-b5c0-fbb2e62f56f9 nodeName:}" failed. No retries permitted until 2026-09-19 10:36:36.494597718 +0000 UTC m=+5.990903801 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs") pod "network-metrics-daemon-7r6w2" (UID: "64bed023-f7cc-46f8-b5c0-fbb2e62f56f9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:36:34.595220 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:34.595097    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret\") pod \"global-pull-secret-syncer-75sfv\" (UID: \"acbcb3f6-b65e-4ee4-b497-e5c913e646d7\") " pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:34.595376 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:34.595244    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 10:36:34.595376 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:34.595335    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret podName:acbcb3f6-b65e-4ee4-b497-e5c913e646d7 nodeName:}" failed. No retries permitted until 2026-09-19 10:36:36.595312598 +0000 UTC m=+6.091618675 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret") pod "global-pull-secret-syncer-75sfv" (UID: "acbcb3f6-b65e-4ee4-b497-e5c913e646d7") : object "kube-system"/"original-pull-secret" not registered
Sep 19 10:36:34.696674 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:34.696555    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgsj\" (UniqueName: \"kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj\") pod \"network-check-target-n7g7l\" (UID: \"b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4\") " pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:34.696845 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:34.696751    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 10:36:34.696845 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:34.696779    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 10:36:34.696845 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:34.696793    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wmgsj for pod openshift-network-diagnostics/network-check-target-n7g7l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:36:34.696995 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:34.696858    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj podName:b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4 nodeName:}" failed. No retries permitted until 2026-09-19 10:36:36.696838322 +0000 UTC m=+6.193144411 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wmgsj" (UniqueName: "kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj") pod "network-check-target-n7g7l" (UID: "b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:36:34.909273 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:34.909165    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 10:31:32 +0000 UTC" deadline="2028-05-15 11:19:10.091529014 +0000 UTC"
Sep 19 10:36:34.909273 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:34.909272    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14496h42m35.182262112s"
Sep 19 10:36:34.996258 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:34.995673    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:34.996258 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:34.995683    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:34.996258 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:34.995810    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7g7l" podUID="b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4"
Sep 19 10:36:34.996258 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:34.995893    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75sfv" podUID="acbcb3f6-b65e-4ee4-b497-e5c913e646d7"
Sep 19 10:36:34.996258 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:34.995970    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:34.996258 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:34.996075    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r6w2" podUID="64bed023-f7cc-46f8-b5c0-fbb2e62f56f9"
Sep 19 10:36:35.009842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:35.009802    2576 generic.go:356] "Generic (PLEG): container finished" podID="5859d74372842ffe42460fcd65d85cf5" containerID="d2b016286fde3792f12a3a04f04d0adc114731f8fe13c2637a36e55b7cb5abc2" exitCode=0
Sep 19 10:36:35.010021 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:35.009899    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal" event={"ID":"5859d74372842ffe42460fcd65d85cf5","Type":"ContainerDied","Data":"d2b016286fde3792f12a3a04f04d0adc114731f8fe13c2637a36e55b7cb5abc2"}
Sep 19 10:36:35.014627 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:35.014593    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jvnt2" event={"ID":"3bda56d0-e1c8-42c2-8905-9719bd0eace8","Type":"ContainerStarted","Data":"8d44b45279211a9d0c297aa48f4f8c886c2d6835a974a1a14f06845ab417e4b8"}
Sep 19 10:36:35.026758 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:35.026659    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vscrj" event={"ID":"a3b17f10-243c-4741-ad8d-feb11c375cbb","Type":"ContainerStarted","Data":"f26f34f4690b5bb790c7b230c22412ee3310ea4ded192b7f609d22f1dfa49679"}
Sep 19 10:36:35.039241 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:35.039170    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vkrwn" event={"ID":"1ec6f61d-3386-407c-8be3-9ba0dfe3045d","Type":"ContainerStarted","Data":"574b4881dba2623e45eae567c4a76ac9d52068219c8879e465dae69122ae2c8e"}
Sep 19 10:36:35.061115 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:35.060853    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s5hpc" event={"ID":"a48975ea-c138-4925-9447-74c6df9a3c36","Type":"ContainerStarted","Data":"021ec1711630a9e80af1434c3a0e4cfcf9254a0f758346eacadeca664b4236af"}
Sep 19 10:36:35.080778 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:35.080735    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z6nh5" event={"ID":"e10a7fca-fab2-40fe-888c-425f875e9a3e","Type":"ContainerStarted","Data":"466a659b4859669667a2012022621d5d10928e0b2c7ba9f24db8e6952619ced5"}
Sep 19 10:36:35.103325 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:35.103238    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal" event={"ID":"a889b3717fa0a5037e893154459d9772","Type":"ContainerStarted","Data":"c80a27888c7dff21c844cad32ec20a6fb45ae7fb00f9296e1b869f6c640829af"}
Sep 19 10:36:35.123054 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:35.122995    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w" event={"ID":"eafc0cde-0b81-4e6e-83b9-3967ed105371","Type":"ContainerStarted","Data":"42c36544a805ce3c125b6838037dc1f171708c1938a21abd039727940bb1ee97"}
Sep 19 10:36:35.136722 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:35.136681    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zbwz5" event={"ID":"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d","Type":"ContainerStarted","Data":"63edc5133b8a6747e3de6bf7f741674b91f0984f40c977c378e14e761074a2cb"}
Sep 19 10:36:35.153171 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:35.153093    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5f6pp" event={"ID":"e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d","Type":"ContainerStarted","Data":"62fbff6018d3aeb056a1b40a8537815abb0792d9d7d42fd5aa6e49c42bc4e743"}
Sep 19 10:36:35.165415 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:35.165143    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2" event={"ID":"96999fca-94e0-4248-b251-eb47595cfd07","Type":"ContainerStarted","Data":"7c4c11ed41dbc490ee29218c288c6a0a62e0244d365b72162eef975979fd4066"}
Sep 19 10:36:36.191278 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:36.191119    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal" event={"ID":"5859d74372842ffe42460fcd65d85cf5","Type":"ContainerStarted","Data":"a0e19480727d382e8d42764a2b6fbaa0aa6a1e6a4b0b5f9860db73ff434d2bbc"}
Sep 19 10:36:36.208455 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:36.208375    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal" podStartSLOduration=4.208349925 podStartE2EDuration="4.208349925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:36:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:36:35.118966376 +0000 UTC m=+4.615272469" watchObservedRunningTime="2026-09-19 10:36:36.208349925 +0000 UTC m=+5.704656018"
Sep 19 10:36:36.510134 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:36.510028    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs\") pod \"network-metrics-daemon-7r6w2\" (UID: \"64bed023-f7cc-46f8-b5c0-fbb2e62f56f9\") " pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:36.510297 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:36.510210    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:36:36.510297 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:36.510277    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs podName:64bed023-f7cc-46f8-b5c0-fbb2e62f56f9 nodeName:}" failed. No retries permitted until 2026-09-19 10:36:40.510258032 +0000 UTC m=+10.006564117 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs") pod "network-metrics-daemon-7r6w2" (UID: "64bed023-f7cc-46f8-b5c0-fbb2e62f56f9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:36:36.611279 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:36.611237    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret\") pod \"global-pull-secret-syncer-75sfv\" (UID: \"acbcb3f6-b65e-4ee4-b497-e5c913e646d7\") " pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:36.611485 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:36.611384    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 10:36:36.611485 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:36.611451    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret podName:acbcb3f6-b65e-4ee4-b497-e5c913e646d7 nodeName:}" failed. No retries permitted until 2026-09-19 10:36:40.611432362 +0000 UTC m=+10.107738448 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret") pod "global-pull-secret-syncer-75sfv" (UID: "acbcb3f6-b65e-4ee4-b497-e5c913e646d7") : object "kube-system"/"original-pull-secret" not registered
Sep 19 10:36:36.712142 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:36.712094    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgsj\" (UniqueName: \"kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj\") pod \"network-check-target-n7g7l\" (UID: \"b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4\") " pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:36.712629 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:36.712601    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 10:36:36.712629 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:36.712639    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 10:36:36.712813 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:36.712657    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wmgsj for pod openshift-network-diagnostics/network-check-target-n7g7l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:36:36.712813 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:36.712735    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj podName:b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4 nodeName:}" failed. No retries permitted until 2026-09-19 10:36:40.71271223 +0000 UTC m=+10.209018310 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wmgsj" (UniqueName: "kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj") pod "network-check-target-n7g7l" (UID: "b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:36:36.996479 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:36.995690    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:36.996479 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:36.995840    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r6w2" podUID="64bed023-f7cc-46f8-b5c0-fbb2e62f56f9"
Sep 19 10:36:36.996479 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:36.996265    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:36.996479 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:36.996361    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7g7l" podUID="b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4"
Sep 19 10:36:36.997012 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:36.996879    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:36.997012 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:36.996975    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75sfv" podUID="acbcb3f6-b65e-4ee4-b497-e5c913e646d7"
Sep 19 10:36:38.996194 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:38.996159    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:38.996194 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:38.996196    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:38.996722 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:38.996159    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:38.996722 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:38.996304    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75sfv" podUID="acbcb3f6-b65e-4ee4-b497-e5c913e646d7"
Sep 19 10:36:38.996722 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:38.996347    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7g7l" podUID="b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4"
Sep 19 10:36:38.996722 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:38.996419    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r6w2" podUID="64bed023-f7cc-46f8-b5c0-fbb2e62f56f9"
Sep 19 10:36:40.543867 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:40.543781    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs\") pod \"network-metrics-daemon-7r6w2\" (UID: \"64bed023-f7cc-46f8-b5c0-fbb2e62f56f9\") " pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:40.544270 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:40.543963    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:36:40.544270 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:40.544055    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs podName:64bed023-f7cc-46f8-b5c0-fbb2e62f56f9 nodeName:}" failed. No retries permitted until 2026-09-19 10:36:48.544015957 +0000 UTC m=+18.040322044 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs") pod "network-metrics-daemon-7r6w2" (UID: "64bed023-f7cc-46f8-b5c0-fbb2e62f56f9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:36:40.645363 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:40.645307    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret\") pod \"global-pull-secret-syncer-75sfv\" (UID: \"acbcb3f6-b65e-4ee4-b497-e5c913e646d7\") " pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:40.645560 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:40.645510    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 10:36:40.645630 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:40.645576    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret podName:acbcb3f6-b65e-4ee4-b497-e5c913e646d7 nodeName:}" failed. No retries permitted until 2026-09-19 10:36:48.645557905 +0000 UTC m=+18.141863998 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret") pod "global-pull-secret-syncer-75sfv" (UID: "acbcb3f6-b65e-4ee4-b497-e5c913e646d7") : object "kube-system"/"original-pull-secret" not registered
Sep 19 10:36:40.746575 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:40.745939    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgsj\" (UniqueName: \"kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj\") pod \"network-check-target-n7g7l\" (UID: \"b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4\") " pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:40.746575 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:40.746126    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 10:36:40.746575 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:40.746150    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 10:36:40.746575 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:40.746163    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wmgsj for pod openshift-network-diagnostics/network-check-target-n7g7l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:36:40.746575 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:40.746225    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj podName:b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4 nodeName:}" failed. No retries permitted until 2026-09-19 10:36:48.746205692 +0000 UTC m=+18.242511764 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wmgsj" (UniqueName: "kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj") pod "network-check-target-n7g7l" (UID: "b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:36:41.002537 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:41.002453    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:41.002537 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:41.002499    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:41.002752 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:41.002587    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75sfv" podUID="acbcb3f6-b65e-4ee4-b497-e5c913e646d7"
Sep 19 10:36:41.002752 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:41.002667    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r6w2" podUID="64bed023-f7cc-46f8-b5c0-fbb2e62f56f9"
Sep 19 10:36:41.002752 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:41.002737    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:41.002874 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:41.002807    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7g7l" podUID="b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4"
Sep 19 10:36:42.998986 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:42.998954    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:42.999545 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:42.999033    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:42.999545 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:42.999065    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:42.999545 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:42.999165    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7g7l" podUID="b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4"
Sep 19 10:36:42.999545 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:42.999307    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75sfv" podUID="acbcb3f6-b65e-4ee4-b497-e5c913e646d7"
Sep 19 10:36:42.999545 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:42.999421    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r6w2" podUID="64bed023-f7cc-46f8-b5c0-fbb2e62f56f9"
Sep 19 10:36:44.995719 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:44.995675    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:44.996222 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:44.995675    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:44.996222 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:44.995796    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75sfv" podUID="acbcb3f6-b65e-4ee4-b497-e5c913e646d7"
Sep 19 10:36:44.996222 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:44.995854    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7g7l" podUID="b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4"
Sep 19 10:36:44.996222 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:44.995686    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:44.996222 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:44.995994    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r6w2" podUID="64bed023-f7cc-46f8-b5c0-fbb2e62f56f9"
Sep 19 10:36:46.995105 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:46.995057    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:46.995594 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:46.995174    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7g7l" podUID="b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4"
Sep 19 10:36:46.995594 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:46.995471    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:46.995594 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:46.995556    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75sfv" podUID="acbcb3f6-b65e-4ee4-b497-e5c913e646d7"
Sep 19 10:36:46.995768 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:46.995611    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:46.995768 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:46.995665    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r6w2" podUID="64bed023-f7cc-46f8-b5c0-fbb2e62f56f9"
Sep 19 10:36:48.610494 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:48.610458    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs\") pod \"network-metrics-daemon-7r6w2\" (UID: \"64bed023-f7cc-46f8-b5c0-fbb2e62f56f9\") " pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:48.611024 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:48.610618    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:36:48.611024 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:48.610694    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs podName:64bed023-f7cc-46f8-b5c0-fbb2e62f56f9 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:04.610677978 +0000 UTC m=+34.106984059 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs") pod "network-metrics-daemon-7r6w2" (UID: "64bed023-f7cc-46f8-b5c0-fbb2e62f56f9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:36:48.710857 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:48.710818    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret\") pod \"global-pull-secret-syncer-75sfv\" (UID: \"acbcb3f6-b65e-4ee4-b497-e5c913e646d7\") " pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:48.711027 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:48.710955    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 10:36:48.711125 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:48.711032    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret podName:acbcb3f6-b65e-4ee4-b497-e5c913e646d7 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:04.711014142 +0000 UTC m=+34.207320225 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret") pod "global-pull-secret-syncer-75sfv" (UID: "acbcb3f6-b65e-4ee4-b497-e5c913e646d7") : object "kube-system"/"original-pull-secret" not registered
Sep 19 10:36:48.812162 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:48.812123    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgsj\" (UniqueName: \"kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj\") pod \"network-check-target-n7g7l\" (UID: \"b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4\") " pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:48.813138 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:48.812297    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 10:36:48.813138 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:48.812329    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 10:36:48.813138 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:48.812344    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wmgsj for pod openshift-network-diagnostics/network-check-target-n7g7l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:36:48.813138 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:48.812401    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj podName:b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:04.812387113 +0000 UTC m=+34.308693186 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wmgsj" (UniqueName: "kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj") pod "network-check-target-n7g7l" (UID: "b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:36:48.995814 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:48.995302    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:48.995814 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:48.995340    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:48.995814 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:48.995305    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:48.995814 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:48.995438    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7g7l" podUID="b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4"
Sep 19 10:36:48.995814 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:48.995562    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r6w2" podUID="64bed023-f7cc-46f8-b5c0-fbb2e62f56f9"
Sep 19 10:36:48.995814 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:48.995655    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75sfv" podUID="acbcb3f6-b65e-4ee4-b497-e5c913e646d7"
Sep 19 10:36:50.996964 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:50.996924    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:50.997466 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:50.997014    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:50.997466 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:50.997055    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7g7l" podUID="b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4"
Sep 19 10:36:50.997466 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:50.997165    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75sfv" podUID="acbcb3f6-b65e-4ee4-b497-e5c913e646d7"
Sep 19 10:36:50.997466 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:50.997226    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:50.997466 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:50.997310    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r6w2" podUID="64bed023-f7cc-46f8-b5c0-fbb2e62f56f9"
Sep 19 10:36:52.995452 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:52.995282    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:52.996245 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:52.995282    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:52.996245 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:52.995566    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7g7l" podUID="b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4"
Sep 19 10:36:52.996245 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:52.995335    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:52.996245 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:52.995630    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r6w2" podUID="64bed023-f7cc-46f8-b5c0-fbb2e62f56f9"
Sep 19 10:36:52.996245 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:52.995736    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75sfv" podUID="acbcb3f6-b65e-4ee4-b497-e5c913e646d7"
Sep 19 10:36:53.226284 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.226188    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jvnt2" event={"ID":"3bda56d0-e1c8-42c2-8905-9719bd0eace8","Type":"ContainerStarted","Data":"b5c7e5c1d2849b2738938cd049512e41bac48bbc0cc205e498aaf98895dec9bd"}
Sep 19 10:36:53.228175 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.228139    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vscrj" event={"ID":"a3b17f10-243c-4741-ad8d-feb11c375cbb","Type":"ContainerStarted","Data":"faeadf982be98bf16cb07135d9eb95f5ae8c55eef2a735c669a5919e2c3ab371"}
Sep 19 10:36:53.229855 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.229825    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vkrwn" event={"ID":"1ec6f61d-3386-407c-8be3-9ba0dfe3045d","Type":"ContainerStarted","Data":"0a6654f14b7f6146f8cb333ecce47dd9c6d201b08cc674e510c1e044940e6b2f"}
Sep 19 10:36:53.231569 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.231544    2576 generic.go:356] "Generic (PLEG): container finished" podID="e10a7fca-fab2-40fe-888c-425f875e9a3e" containerID="f6718f2c3de344b2a51ba681e8b6d1f83c80b600a2cdf5be524d9e5ec49c2b04" exitCode=0
Sep 19 10:36:53.231711 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.231628    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z6nh5" event={"ID":"e10a7fca-fab2-40fe-888c-425f875e9a3e","Type":"ContainerDied","Data":"f6718f2c3de344b2a51ba681e8b6d1f83c80b600a2cdf5be524d9e5ec49c2b04"}
Sep 19 10:36:53.235640 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.235610    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w" event={"ID":"eafc0cde-0b81-4e6e-83b9-3967ed105371","Type":"ContainerStarted","Data":"435b5e109b9c7ca9334afcfb62b9ed9ee0ce4bffcec99062fc1812943a688552"}
Sep 19 10:36:53.235865 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.235703    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w" event={"ID":"eafc0cde-0b81-4e6e-83b9-3967ed105371","Type":"ContainerStarted","Data":"c750d52e6acf8b33ba2896a1202a57da49256207f4dfe98f40e00ea9a0b6e868"}
Sep 19 10:36:53.235865 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.235760    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w" event={"ID":"eafc0cde-0b81-4e6e-83b9-3967ed105371","Type":"ContainerStarted","Data":"ea0dc18733d5312fc71b87a56183881bfb370c4da232b8456f6688a6f35e4492"}
Sep 19 10:36:53.235865 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.235806    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w" event={"ID":"eafc0cde-0b81-4e6e-83b9-3967ed105371","Type":"ContainerStarted","Data":"a1a8e3b0260a6eb6ee5053b33515bd35aeede2f15bfaea9087a30d1c6fd3a712"}
Sep 19 10:36:53.235865 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.235856    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w" event={"ID":"eafc0cde-0b81-4e6e-83b9-3967ed105371","Type":"ContainerStarted","Data":"d4da629b1f695a1ebb5f506cd18ed0df57b8b071f013af48f4de6bd545f50eb2"}
Sep 19 10:36:53.236123 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.235894    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w" event={"ID":"eafc0cde-0b81-4e6e-83b9-3967ed105371","Type":"ContainerStarted","Data":"a4e184e853010dbe07dbd69532482d55842f11b87c4e8c92cc60a0b3f557ed9a"}
Sep 19 10:36:53.237379 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.237349    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zbwz5" event={"ID":"5f7d5cb0-8c94-4757-967a-dbd6c59fc53d","Type":"ContainerStarted","Data":"5175d9fe9979755a37dd09b988a51452167ca88b5e19dde33cc2a42dc5e80309"}
Sep 19 10:36:53.239574 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.239540    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5f6pp" event={"ID":"e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d","Type":"ContainerStarted","Data":"5052a76ef032e9094422e86c160e9e7d4eb6863f37cf91de280407784fe4deb3"}
Sep 19 10:36:53.239794 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.239738    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jvnt2" podStartSLOduration=4.023573108 podStartE2EDuration="22.239721125s" totalImagesPullingTime="18.216148017s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:36:53.239407227 +0000 UTC m=+22.735713319" watchObservedRunningTime="2026-09-19 10:36:53.239721125 +0000 UTC m=+22.736027231"
Sep 19 10:36:53.239865 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.239823    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal" podStartSLOduration=21.239816109 podStartE2EDuration="21.239816109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:36:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:36:36.209002145 +0000 UTC m=+5.705308238" watchObservedRunningTime="2026-09-19 10:36:53.239816109 +0000 UTC m=+22.736122203"
Sep 19 10:36:53.240964 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.240940    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2" event={"ID":"96999fca-94e0-4248-b251-eb47595cfd07","Type":"ContainerStarted","Data":"a5e66a7f7387698482f2ae0ffef58b20e453ff71f2ce8fa64735ac7c4fe19ce8"}
Sep 19 10:36:53.253842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.253774    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5f6pp" podStartSLOduration=4.030986163 podStartE2EDuration="22.253740704s" totalImagesPullingTime="18.222754541s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:36:53.252759049 +0000 UTC m=+22.749065142" watchObservedRunningTime="2026-09-19 10:36:53.253740704 +0000 UTC m=+22.750046798"
Sep 19 10:36:53.269406 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.269348    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zbwz5" podStartSLOduration=4.021102298 podStartE2EDuration="22.269331078s" totalImagesPullingTime="18.24822878s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:36:53.26872931 +0000 UTC m=+22.765035412" watchObservedRunningTime="2026-09-19 10:36:53.269331078 +0000 UTC m=+22.765637169"
Sep 19 10:36:53.284214 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.284157    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vscrj" podStartSLOduration=4.000335786 podStartE2EDuration="22.284143477s" totalImagesPullingTime="18.283807691s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:36:53.283919025 +0000 UTC m=+22.780225117" watchObservedRunningTime="2026-09-19 10:36:53.284143477 +0000 UTC m=+22.780449583"
Sep 19 10:36:53.866562 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.866373    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 19 10:36:53.933420 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.933315    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-19T10:36:53.866559747Z","UUID":"a01bdbbc-7498-4bb5-b8fa-b39b3a8236ff","Handler":null,"Name":"","Endpoint":""}
Sep 19 10:36:53.934792 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.934771    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 19 10:36:53.934910 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:53.934801    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 19 10:36:54.246145 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:54.246102    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2" event={"ID":"96999fca-94e0-4248-b251-eb47595cfd07","Type":"ContainerStarted","Data":"d1c0986bb38023dee5972eefa3602a4cca3e2f1212a1b622c57d066f9edb53bb"}
Sep 19 10:36:54.247979 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:54.247944    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s5hpc" event={"ID":"a48975ea-c138-4925-9447-74c6df9a3c36","Type":"ContainerStarted","Data":"6c12d207887a309fcc86eb045326acf0429e622cec3cbbb32d2cd0f766a54fcb"}
Sep 19 10:36:54.261993 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:54.261942    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vkrwn" podStartSLOduration=8.384353903 podStartE2EDuration="23.26192193s" totalImagesPullingTime="14.877568027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:36:53.315768113 +0000 UTC m=+22.812074206" watchObservedRunningTime="2026-09-19 10:36:54.26192193 +0000 UTC m=+23.758228022"
Sep 19 10:36:54.262329 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:54.262305    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-s5hpc" podStartSLOduration=5.038664397 podStartE2EDuration="23.262297614s" totalImagesPullingTime="18.223633217s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:36:54.261684798 +0000 UTC m=+23.757990891" watchObservedRunningTime="2026-09-19 10:36:54.262297614 +0000 UTC m=+23.758603705"
Sep 19 10:36:54.995749 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:54.995704    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:54.995749 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:54.995744    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:54.995749 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:54.995751    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:54.996069 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:54.995840    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7g7l" podUID="b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4"
Sep 19 10:36:54.996069 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:54.995932    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75sfv" podUID="acbcb3f6-b65e-4ee4-b497-e5c913e646d7"
Sep 19 10:36:54.996069 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:54.996034    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r6w2" podUID="64bed023-f7cc-46f8-b5c0-fbb2e62f56f9"
Sep 19 10:36:55.254250 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:55.254154    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w" event={"ID":"eafc0cde-0b81-4e6e-83b9-3967ed105371","Type":"ContainerStarted","Data":"7a870b0f09c74b181b2c32d5bffd51c3060af733a235345e2739071795ed1657"}
Sep 19 10:36:55.256495 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:55.256459    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2" event={"ID":"96999fca-94e0-4248-b251-eb47595cfd07","Type":"ContainerStarted","Data":"2533d11972da769d47e1f7b6de0f002fe91288cd0ea73b0edfe639ecbc0d93ec"}
Sep 19 10:36:55.276550 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:55.276488    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mkcm2" podStartSLOduration=3.793355925 podStartE2EDuration="24.276470919s" totalImagesPullingTime="20.483114994s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:36:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:36:55.274773615 +0000 UTC m=+24.771079708" watchObservedRunningTime="2026-09-19 10:36:55.276470919 +0000 UTC m=+24.772777008"
Sep 19 10:36:56.971404 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:56.971366    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vkrwn"
Sep 19 10:36:56.972016 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:56.971988    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vkrwn"
Sep 19 10:36:56.996080 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:56.996025    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:56.996247 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:56.996186    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r6w2" podUID="64bed023-f7cc-46f8-b5c0-fbb2e62f56f9"
Sep 19 10:36:56.996555 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:56.996535    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:56.996630 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:56.996611    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7g7l" podUID="b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4"
Sep 19 10:36:56.996716 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:56.996696    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:56.996835 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:56.996815    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75sfv" podUID="acbcb3f6-b65e-4ee4-b497-e5c913e646d7"
Sep 19 10:36:57.262405 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:57.262370    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vkrwn"
Sep 19 10:36:57.262916 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:57.262892    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vkrwn"
Sep 19 10:36:58.995889 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:58.995720    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:36:58.996322 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:58.995806    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:36:58.996322 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:58.995979    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75sfv" podUID="acbcb3f6-b65e-4ee4-b497-e5c913e646d7"
Sep 19 10:36:58.996322 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:58.995831    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:36:58.996322 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:58.996067    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r6w2" podUID="64bed023-f7cc-46f8-b5c0-fbb2e62f56f9"
Sep 19 10:36:58.996322 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:36:58.996150    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7g7l" podUID="b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4"
Sep 19 10:36:59.268416 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:59.268376    2576 generic.go:356] "Generic (PLEG): container finished" podID="e10a7fca-fab2-40fe-888c-425f875e9a3e" containerID="e8bd14e4865448f0386a6c18472a492f3324bb48a8e07d3fdbd6e24b1e174abe" exitCode=0
Sep 19 10:36:59.268576 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:59.268461    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z6nh5" event={"ID":"e10a7fca-fab2-40fe-888c-425f875e9a3e","Type":"ContainerDied","Data":"e8bd14e4865448f0386a6c18472a492f3324bb48a8e07d3fdbd6e24b1e174abe"}
Sep 19 10:36:59.272500 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:59.272475    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w" event={"ID":"eafc0cde-0b81-4e6e-83b9-3967ed105371","Type":"ContainerStarted","Data":"942226ea7c5f270362c03e866c0f5baddcced8bfe6968bcae785aee264189f2e"}
Sep 19 10:36:59.360421 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:36:59.360312    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w" podStartSLOduration=10.062573977 podStartE2EDuration="28.360293761s" totalImagesPullingTime="18.297719784s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:36:59.358560068 +0000 UTC m=+28.854866159" watchObservedRunningTime="2026-09-19 10:36:59.360293761 +0000 UTC m=+28.856599874"
Sep 19 10:37:00.275109 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:00.275077    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:37:00.275693 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:00.275175    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:37:00.275693 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:00.275228    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:37:00.292975 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:00.292947    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:37:00.293146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:00.293130    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:37:00.554592 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:00.554197    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-75sfv"]
Sep 19 10:37:00.554895 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:00.554838    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:37:00.559086 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:00.555129    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75sfv" podUID="acbcb3f6-b65e-4ee4-b497-e5c913e646d7"
Sep 19 10:37:00.559086 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:00.556207    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n7g7l"]
Sep 19 10:37:00.559086 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:00.556347    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:37:00.559086 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:00.556473    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7g7l" podUID="b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4"
Sep 19 10:37:00.560789 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:00.560764    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7r6w2"]
Sep 19 10:37:00.560905 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:00.560858    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:37:00.560995 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:00.560969    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r6w2" podUID="64bed023-f7cc-46f8-b5c0-fbb2e62f56f9"
Sep 19 10:37:01.278701 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:01.278661    2576 generic.go:356] "Generic (PLEG): container finished" podID="e10a7fca-fab2-40fe-888c-425f875e9a3e" containerID="eaa66bf34e45b70e3758b94d0e84a39ca3c5cf135b28548144c92584c7c1b65a" exitCode=0
Sep 19 10:37:01.279266 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:01.278742    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z6nh5" event={"ID":"e10a7fca-fab2-40fe-888c-425f875e9a3e","Type":"ContainerDied","Data":"eaa66bf34e45b70e3758b94d0e84a39ca3c5cf135b28548144c92584c7c1b65a"}
Sep 19 10:37:01.995112 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:01.995080    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:37:01.995276 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:01.995136    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:37:01.995276 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:01.995086    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:37:01.995276 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:01.995216    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75sfv" podUID="acbcb3f6-b65e-4ee4-b497-e5c913e646d7"
Sep 19 10:37:01.995440 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:01.995300    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r6w2" podUID="64bed023-f7cc-46f8-b5c0-fbb2e62f56f9"
Sep 19 10:37:01.995440 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:01.995377    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7g7l" podUID="b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4"
Sep 19 10:37:02.284375 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:02.284290    2576 generic.go:356] "Generic (PLEG): container finished" podID="e10a7fca-fab2-40fe-888c-425f875e9a3e" containerID="55fa970af6e3d22bcacb90d5be2e3094191f65462e849cb16b4aaaac33ec2b5b" exitCode=0
Sep 19 10:37:02.284734 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:02.284377    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z6nh5" event={"ID":"e10a7fca-fab2-40fe-888c-425f875e9a3e","Type":"ContainerDied","Data":"55fa970af6e3d22bcacb90d5be2e3094191f65462e849cb16b4aaaac33ec2b5b"}
Sep 19 10:37:03.995992 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:03.995947    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:37:03.996567 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:03.995962    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:37:03.996567 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:03.996127    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7r6w2" podUID="64bed023-f7cc-46f8-b5c0-fbb2e62f56f9"
Sep 19 10:37:03.996567 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:03.995972    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:37:03.996567 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:03.996156    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-75sfv" podUID="acbcb3f6-b65e-4ee4-b497-e5c913e646d7"
Sep 19 10:37:03.996567 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:03.996238    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7g7l" podUID="b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4"
Sep 19 10:37:04.635795 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:04.635603    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs\") pod \"network-metrics-daemon-7r6w2\" (UID: \"64bed023-f7cc-46f8-b5c0-fbb2e62f56f9\") " pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:37:04.635969 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:04.635764    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:37:04.635969 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:04.635870    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs podName:64bed023-f7cc-46f8-b5c0-fbb2e62f56f9 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:36.63585191 +0000 UTC m=+66.132157998 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs") pod "network-metrics-daemon-7r6w2" (UID: "64bed023-f7cc-46f8-b5c0-fbb2e62f56f9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:37:04.737106 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:04.737066    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret\") pod \"global-pull-secret-syncer-75sfv\" (UID: \"acbcb3f6-b65e-4ee4-b497-e5c913e646d7\") " pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:37:04.737301 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:04.737231    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 10:37:04.737370 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:04.737313    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret podName:acbcb3f6-b65e-4ee4-b497-e5c913e646d7 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:36.737291551 +0000 UTC m=+66.233597626 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret") pod "global-pull-secret-syncer-75sfv" (UID: "acbcb3f6-b65e-4ee4-b497-e5c913e646d7") : object "kube-system"/"original-pull-secret" not registered
Sep 19 10:37:04.837652 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:04.837606    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgsj\" (UniqueName: \"kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj\") pod \"network-check-target-n7g7l\" (UID: \"b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4\") " pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:37:04.837843 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:04.837772    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 10:37:04.837843 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:04.837792    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 10:37:04.837843 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:04.837805    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wmgsj for pod openshift-network-diagnostics/network-check-target-n7g7l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:37:04.838012 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:04.837885    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj podName:b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:36.837866521 +0000 UTC m=+66.334172608 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wmgsj" (UniqueName: "kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj") pod "network-check-target-n7g7l" (UID: "b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:37:05.249597 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.249515    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeReady"
Sep 19 10:37:05.250031 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.249652    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 19 10:37:05.296880 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.296847    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wf45t"]
Sep 19 10:37:05.299021 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.298993    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:05.299382 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.299352    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-g5qhx"]
Sep 19 10:37:05.301153 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.301131    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g5qhx"
Sep 19 10:37:05.301862 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.301839    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 19 10:37:05.301968 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.301871    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qlvz2\""
Sep 19 10:37:05.301968 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.301871    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 19 10:37:05.303680 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.303659    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 19 10:37:05.303680 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.303672    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 19 10:37:05.303852 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.303710    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-62jc5\""
Sep 19 10:37:05.303852 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.303659    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 19 10:37:05.312779 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.311118    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wf45t"]
Sep 19 10:37:05.314306 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.314283    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g5qhx"]
Sep 19 10:37:05.442305 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.442265    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwmt7\" (UniqueName: \"kubernetes.io/projected/bd12ed6b-5433-4b19-9eba-1973e0bd4afa-kube-api-access-bwmt7\") pod \"ingress-canary-g5qhx\" (UID: \"bd12ed6b-5433-4b19-9eba-1973e0bd4afa\") " pod="openshift-ingress-canary/ingress-canary-g5qhx"
Sep 19 10:37:05.442506 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.442314    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55t6p\" (UniqueName: \"kubernetes.io/projected/daad582d-23b8-445c-8c99-dec7c11c95a7-kube-api-access-55t6p\") pod \"dns-default-wf45t\" (UID: \"daad582d-23b8-445c-8c99-dec7c11c95a7\") " pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:05.442506 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.442371    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd12ed6b-5433-4b19-9eba-1973e0bd4afa-cert\") pod \"ingress-canary-g5qhx\" (UID: \"bd12ed6b-5433-4b19-9eba-1973e0bd4afa\") " pod="openshift-ingress-canary/ingress-canary-g5qhx"
Sep 19 10:37:05.442506 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.442455    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daad582d-23b8-445c-8c99-dec7c11c95a7-tmp-dir\") pod \"dns-default-wf45t\" (UID: \"daad582d-23b8-445c-8c99-dec7c11c95a7\") " pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:05.442624 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.442508    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/daad582d-23b8-445c-8c99-dec7c11c95a7-config-volume\") pod \"dns-default-wf45t\" (UID: \"daad582d-23b8-445c-8c99-dec7c11c95a7\") " pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:05.442624 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.442601    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls\") pod \"dns-default-wf45t\" (UID: \"daad582d-23b8-445c-8c99-dec7c11c95a7\") " pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:05.543619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.543517    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls\") pod \"dns-default-wf45t\" (UID: \"daad582d-23b8-445c-8c99-dec7c11c95a7\") " pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:05.543619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.543572    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwmt7\" (UniqueName: \"kubernetes.io/projected/bd12ed6b-5433-4b19-9eba-1973e0bd4afa-kube-api-access-bwmt7\") pod \"ingress-canary-g5qhx\" (UID: \"bd12ed6b-5433-4b19-9eba-1973e0bd4afa\") " pod="openshift-ingress-canary/ingress-canary-g5qhx"
Sep 19 10:37:05.543619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.543604    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55t6p\" (UniqueName: \"kubernetes.io/projected/daad582d-23b8-445c-8c99-dec7c11c95a7-kube-api-access-55t6p\") pod \"dns-default-wf45t\" (UID: \"daad582d-23b8-445c-8c99-dec7c11c95a7\") " pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:05.543898 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.543637    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd12ed6b-5433-4b19-9eba-1973e0bd4afa-cert\") pod \"ingress-canary-g5qhx\" (UID: \"bd12ed6b-5433-4b19-9eba-1973e0bd4afa\") " pod="openshift-ingress-canary/ingress-canary-g5qhx"
Sep 19 10:37:05.543898 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.543663    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daad582d-23b8-445c-8c99-dec7c11c95a7-tmp-dir\") pod \"dns-default-wf45t\" (UID: \"daad582d-23b8-445c-8c99-dec7c11c95a7\") " pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:05.543898 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.543688    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/daad582d-23b8-445c-8c99-dec7c11c95a7-config-volume\") pod \"dns-default-wf45t\" (UID: \"daad582d-23b8-445c-8c99-dec7c11c95a7\") " pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:05.543898 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:05.543695    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 10:37:05.543898 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:05.543786    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls podName:daad582d-23b8-445c-8c99-dec7c11c95a7 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:06.043759861 +0000 UTC m=+35.540065947 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls") pod "dns-default-wf45t" (UID: "daad582d-23b8-445c-8c99-dec7c11c95a7") : secret "dns-default-metrics-tls" not found
Sep 19 10:37:05.544251 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.544228    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daad582d-23b8-445c-8c99-dec7c11c95a7-tmp-dir\") pod \"dns-default-wf45t\" (UID: \"daad582d-23b8-445c-8c99-dec7c11c95a7\") " pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:05.544445 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.544421    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/daad582d-23b8-445c-8c99-dec7c11c95a7-config-volume\") pod \"dns-default-wf45t\" (UID: \"daad582d-23b8-445c-8c99-dec7c11c95a7\") " pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:05.548471 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.548446    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd12ed6b-5433-4b19-9eba-1973e0bd4afa-cert\") pod \"ingress-canary-g5qhx\" (UID: \"bd12ed6b-5433-4b19-9eba-1973e0bd4afa\") " pod="openshift-ingress-canary/ingress-canary-g5qhx"
Sep 19 10:37:05.551987 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.551959    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55t6p\" (UniqueName: \"kubernetes.io/projected/daad582d-23b8-445c-8c99-dec7c11c95a7-kube-api-access-55t6p\") pod \"dns-default-wf45t\" (UID: \"daad582d-23b8-445c-8c99-dec7c11c95a7\") " pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:05.552188 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.552168    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwmt7\" (UniqueName: \"kubernetes.io/projected/bd12ed6b-5433-4b19-9eba-1973e0bd4afa-kube-api-access-bwmt7\") pod \"ingress-canary-g5qhx\" (UID: \"bd12ed6b-5433-4b19-9eba-1973e0bd4afa\") " pod="openshift-ingress-canary/ingress-canary-g5qhx"
Sep 19 10:37:05.623226 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.623185    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g5qhx"
Sep 19 10:37:05.995456 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.995419    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:37:05.995456 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.995443    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:37:05.995689 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.995451    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:37:05.998350 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.998329    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 19 10:37:05.998487 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.998404    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 19 10:37:05.998487 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.998457    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 19 10:37:05.998487 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.998477    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ql7r7\""
Sep 19 10:37:05.999429 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.999331    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 19 10:37:05.999560 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:05.999461    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6z24q\""
Sep 19 10:37:06.048876 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.048838    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls\") pod \"dns-default-wf45t\" (UID: \"daad582d-23b8-445c-8c99-dec7c11c95a7\") " pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:06.049088 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:06.049014    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 10:37:06.049154 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:06.049126    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls podName:daad582d-23b8-445c-8c99-dec7c11c95a7 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:07.049105784 +0000 UTC m=+36.545411877 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls") pod "dns-default-wf45t" (UID: "daad582d-23b8-445c-8c99-dec7c11c95a7") : secret "dns-default-metrics-tls" not found
Sep 19 10:37:06.653834 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.653796    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5nqg9"]
Sep 19 10:37:06.656254 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.656102    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5nqg9"
Sep 19 10:37:06.656600 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.656574    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-m25hk"]
Sep 19 10:37:06.658391 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.658362    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-m25hk"
Sep 19 10:37:06.658873 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.658855    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-59lzp\""
Sep 19 10:37:06.658968 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.658866    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 19 10:37:06.659603 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.659584    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 19 10:37:06.664170 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.663390    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 19 10:37:06.664711 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.664688    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 19 10:37:06.664814 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.664723    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 19 10:37:06.664921 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.664902    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-x2lp7\""
Sep 19 10:37:06.665132 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.665109    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 19 10:37:06.669012 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.668988    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5nqg9"]
Sep 19 10:37:06.670794 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.670773    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 19 10:37:06.671226 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.671201    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-m25hk"]
Sep 19 10:37:06.752161 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.752126    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-6dllh"]
Sep 19 10:37:06.754142 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.754115    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6dllh"
Sep 19 10:37:06.754380 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.754361    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcq95\" (UniqueName: \"kubernetes.io/projected/d526e598-cec6-460b-abcc-e5bee3890d8e-kube-api-access-wcq95\") pod \"volume-data-source-validator-5c4db8bdf9-5nqg9\" (UID: \"d526e598-cec6-460b-abcc-e5bee3890d8e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5nqg9"
Sep 19 10:37:06.754471 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.754397    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/320e460b-f15a-4da5-82eb-f92a0f756180-config\") pod \"console-operator-fd4bb46c-m25hk\" (UID: \"320e460b-f15a-4da5-82eb-f92a0f756180\") " pod="openshift-console-operator/console-operator-fd4bb46c-m25hk"
Sep 19 10:37:06.754471 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.754424    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vq2b\" (UniqueName: \"kubernetes.io/projected/320e460b-f15a-4da5-82eb-f92a0f756180-kube-api-access-7vq2b\") pod \"console-operator-fd4bb46c-m25hk\" (UID: \"320e460b-f15a-4da5-82eb-f92a0f756180\") " pod="openshift-console-operator/console-operator-fd4bb46c-m25hk"
Sep 19 10:37:06.754580 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.754489    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/320e460b-f15a-4da5-82eb-f92a0f756180-serving-cert\") pod \"console-operator-fd4bb46c-m25hk\" (UID: \"320e460b-f15a-4da5-82eb-f92a0f756180\") " pod="openshift-console-operator/console-operator-fd4bb46c-m25hk"
Sep 19 10:37:06.754580 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.754549    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/320e460b-f15a-4da5-82eb-f92a0f756180-trusted-ca\") pod \"console-operator-fd4bb46c-m25hk\" (UID: \"320e460b-f15a-4da5-82eb-f92a0f756180\") " pod="openshift-console-operator/console-operator-fd4bb46c-m25hk"
Sep 19 10:37:06.756527 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.756504    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 19 10:37:06.756804 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.756788    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 19 10:37:06.757171 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.757129    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 19 10:37:06.757401 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.757383    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-472fw\""
Sep 19 10:37:06.757548 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.757386    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 19 10:37:06.760131 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.759244    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"]
Sep 19 10:37:06.765777 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.763151    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f"]
Sep 19 10:37:06.765777 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.763721    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"
Sep 19 10:37:06.767007 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.766808    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 19 10:37:06.767007 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.766925    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 19 10:37:06.767464 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.767436    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6ddf8bf8dd-66fk8"]
Sep 19 10:37:06.767574 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.767558    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f"
Sep 19 10:37:06.768893 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.768765    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4mppq\""
Sep 19 10:37:06.769354 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.769326    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 19 10:37:06.769458 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.769356    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 19 10:37:06.769665 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.769645    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 19 10:37:06.770666 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.770477    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 19 10:37:06.770666 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.770590    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6gjk5\""
Sep 19 10:37:06.770987 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.770967    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-44pw4"]
Sep 19 10:37:06.771104 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.771093    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:06.771740 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.771723    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 19 10:37:06.773662 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.773639    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-6dllh"]
Sep 19 10:37:06.773743 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.773736    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:06.774694 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.774673    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 19 10:37:06.774855 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.774840    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 19 10:37:06.776458 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.775211    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 19 10:37:06.776458 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.775620    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 19 10:37:06.776458 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.775714    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 19 10:37:06.776458 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.775950    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-j7rmx\""
Sep 19 10:37:06.776458 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.776034    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 19 10:37:06.776858 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.776558    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 19 10:37:06.776858 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.776587    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 19 10:37:06.776858 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.776754    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 19 10:37:06.778403 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.778383    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"]
Sep 19 10:37:06.779706 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.779252    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-n2wd4\""
Sep 19 10:37:06.779706 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.779499    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f"]
Sep 19 10:37:06.779706 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.779554    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 19 10:37:06.781132 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.781110    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-44pw4"]
Sep 19 10:37:06.781304 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.781159    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6ddf8bf8dd-66fk8"]
Sep 19 10:37:06.782970 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.782950    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 19 10:37:06.854970 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.854931    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-96xsm"]
Sep 19 10:37:06.855176 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855084    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-stats-auth\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:06.855176 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855116    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dcf226b4-3c7f-40c0-a327-96003c35e314-serving-cert\") pod \"service-ca-operator-57cdc4b768-6dllh\" (UID: \"dcf226b4-3c7f-40c0-a327-96003c35e314\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6dllh"
Sep 19 10:37:06.855176 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855145    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/320e460b-f15a-4da5-82eb-f92a0f756180-trusted-ca\") pod \"console-operator-fd4bb46c-m25hk\" (UID: \"320e460b-f15a-4da5-82eb-f92a0f756180\") " pod="openshift-console-operator/console-operator-fd4bb46c-m25hk"
Sep 19 10:37:06.855347 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855184    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0b67d17d-e9f0-4b5b-a15f-760f130cae93-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-lntx2\" (UID: \"0b67d17d-e9f0-4b5b-a15f-760f130cae93\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"
Sep 19 10:37:06.855347 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855213    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmpg9\" (UniqueName: \"kubernetes.io/projected/dcf226b4-3c7f-40c0-a327-96003c35e314-kube-api-access-kmpg9\") pod \"service-ca-operator-57cdc4b768-6dllh\" (UID: \"dcf226b4-3c7f-40c0-a327-96003c35e314\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6dllh"
Sep 19 10:37:06.855347 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855236    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2bfbbce9-4493-4ec5-b0d5-d85da7e26e65-snapshots\") pod \"insights-operator-6f9c644dd9-44pw4\" (UID: \"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65\") " pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:06.855347 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855268    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:06.855347 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855291    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f46ws\" (UniqueName: \"kubernetes.io/projected/18495139-9a71-4bd7-b279-1d7e6470271f-kube-api-access-f46ws\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:06.855347 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855325    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgpvf\" (UniqueName: \"kubernetes.io/projected/2bfbbce9-4493-4ec5-b0d5-d85da7e26e65-kube-api-access-xgpvf\") pod \"insights-operator-6f9c644dd9-44pw4\" (UID: \"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65\") " pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:06.855686 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855355    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcq95\" (UniqueName: \"kubernetes.io/projected/d526e598-cec6-460b-abcc-e5bee3890d8e-kube-api-access-wcq95\") pod \"volume-data-source-validator-5c4db8bdf9-5nqg9\" (UID: \"d526e598-cec6-460b-abcc-e5bee3890d8e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5nqg9"
Sep 19 10:37:06.855686 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855379    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmnm8\" (UniqueName: \"kubernetes.io/projected/3755924c-2290-44b0-ad8f-76c6c14c2b27-kube-api-access-nmnm8\") pod \"cluster-samples-operator-685d7d68b5-qzr7f\" (UID: \"3755924c-2290-44b0-ad8f-76c6c14c2b27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f"
Sep 19 10:37:06.855686 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855403    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/320e460b-f15a-4da5-82eb-f92a0f756180-config\") pod \"console-operator-fd4bb46c-m25hk\" (UID: \"320e460b-f15a-4da5-82eb-f92a0f756180\") " pod="openshift-console-operator/console-operator-fd4bb46c-m25hk"
Sep 19 10:37:06.855686 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855427    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/320e460b-f15a-4da5-82eb-f92a0f756180-serving-cert\") pod \"console-operator-fd4bb46c-m25hk\" (UID: \"320e460b-f15a-4da5-82eb-f92a0f756180\") " pod="openshift-console-operator/console-operator-fd4bb46c-m25hk"
Sep 19 10:37:06.855686 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855451    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:06.855686 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855477    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-default-certificate\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:06.855686 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855504    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bfbbce9-4493-4ec5-b0d5-d85da7e26e65-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-44pw4\" (UID: \"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65\") " pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:06.855686 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855550    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bfbbce9-4493-4ec5-b0d5-d85da7e26e65-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-44pw4\" (UID: \"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65\") " pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:06.855686 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855584    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-lntx2\" (UID: \"0b67d17d-e9f0-4b5b-a15f-760f130cae93\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"
Sep 19 10:37:06.855686 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855613    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bfbbce9-4493-4ec5-b0d5-d85da7e26e65-tmp\") pod \"insights-operator-6f9c644dd9-44pw4\" (UID: \"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65\") " pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:06.855686 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855634    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-qzr7f\" (UID: \"3755924c-2290-44b0-ad8f-76c6c14c2b27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f"
Sep 19 10:37:06.855686 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855681    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dcf226b4-3c7f-40c0-a327-96003c35e314-config\") pod \"service-ca-operator-57cdc4b768-6dllh\" (UID: \"dcf226b4-3c7f-40c0-a327-96003c35e314\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6dllh"
Sep 19 10:37:06.856343 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855738    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vq2b\" (UniqueName: \"kubernetes.io/projected/320e460b-f15a-4da5-82eb-f92a0f756180-kube-api-access-7vq2b\") pod \"console-operator-fd4bb46c-m25hk\" (UID: \"320e460b-f15a-4da5-82eb-f92a0f756180\") " pod="openshift-console-operator/console-operator-fd4bb46c-m25hk"
Sep 19 10:37:06.856343 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855762    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2bfbbce9-4493-4ec5-b0d5-d85da7e26e65-serving-cert\") pod \"insights-operator-6f9c644dd9-44pw4\" (UID: \"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65\") " pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:06.856343 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.855802    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrssp\" (UniqueName: \"kubernetes.io/projected/0b67d17d-e9f0-4b5b-a15f-760f130cae93-kube-api-access-xrssp\") pod \"cluster-monitoring-operator-847b8bd8d7-lntx2\" (UID: \"0b67d17d-e9f0-4b5b-a15f-760f130cae93\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"
Sep 19 10:37:06.857220 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.857196    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/320e460b-f15a-4da5-82eb-f92a0f756180-trusted-ca\") pod \"console-operator-fd4bb46c-m25hk\" (UID: \"320e460b-f15a-4da5-82eb-f92a0f756180\") " pod="openshift-console-operator/console-operator-fd4bb46c-m25hk"
Sep 19 10:37:06.857983 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.857959    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/320e460b-f15a-4da5-82eb-f92a0f756180-config\") pod \"console-operator-fd4bb46c-m25hk\" (UID: \"320e460b-f15a-4da5-82eb-f92a0f756180\") " pod="openshift-console-operator/console-operator-fd4bb46c-m25hk"
Sep 19 10:37:06.858385 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.858353    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77fdd969cd-nq42r"]
Sep 19 10:37:06.858481 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.858455    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-96xsm"
Sep 19 10:37:06.859654 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.859636    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/320e460b-f15a-4da5-82eb-f92a0f756180-serving-cert\") pod \"console-operator-fd4bb46c-m25hk\" (UID: \"320e460b-f15a-4da5-82eb-f92a0f756180\") " pod="openshift-console-operator/console-operator-fd4bb46c-m25hk"
Sep 19 10:37:06.860185 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.860168    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:06.860836 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.860818    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 19 10:37:06.860836 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.860829    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 19 10:37:06.861227 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.861211    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 19 10:37:06.861365 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.861345    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-khnrq\""
Sep 19 10:37:06.861462 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.861430    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 19 10:37:06.862393 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.862374    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 19 10:37:06.862505 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.862488    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 19 10:37:06.863143 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.863125    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 19 10:37:06.863636 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.863617    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lp9gc\""
Sep 19 10:37:06.873143 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.873070    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcq95\" (UniqueName: \"kubernetes.io/projected/d526e598-cec6-460b-abcc-e5bee3890d8e-kube-api-access-wcq95\") pod \"volume-data-source-validator-5c4db8bdf9-5nqg9\" (UID: \"d526e598-cec6-460b-abcc-e5bee3890d8e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5nqg9"
Sep 19 10:37:06.873143 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.873101    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-96xsm"]
Sep 19 10:37:06.873424 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.873369    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 19 10:37:06.875666 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.875641    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77fdd969cd-nq42r"]
Sep 19 10:37:06.876805 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.876780    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vq2b\" (UniqueName: \"kubernetes.io/projected/320e460b-f15a-4da5-82eb-f92a0f756180-kube-api-access-7vq2b\") pod \"console-operator-fd4bb46c-m25hk\" (UID: \"320e460b-f15a-4da5-82eb-f92a0f756180\") " pod="openshift-console-operator/console-operator-fd4bb46c-m25hk"
Sep 19 10:37:06.956739 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.956699    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39728838-de41-40c4-8d66-362ab1882457-trusted-ca\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:06.956739 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.956747    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bfbbce9-4493-4ec5-b0d5-d85da7e26e65-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-44pw4\" (UID: \"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65\") " pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:06.956974 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.956771    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-lntx2\" (UID: \"0b67d17d-e9f0-4b5b-a15f-760f130cae93\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"
Sep 19 10:37:06.956974 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.956829    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bfbbce9-4493-4ec5-b0d5-d85da7e26e65-tmp\") pod \"insights-operator-6f9c644dd9-44pw4\" (UID: \"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65\") " pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:06.956974 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.956860    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-qzr7f\" (UID: \"3755924c-2290-44b0-ad8f-76c6c14c2b27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f"
Sep 19 10:37:06.956974 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:06.956866    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 10:37:06.956974 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.956889    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dcf226b4-3c7f-40c0-a327-96003c35e314-config\") pod \"service-ca-operator-57cdc4b768-6dllh\" (UID: \"dcf226b4-3c7f-40c0-a327-96003c35e314\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6dllh"
Sep 19 10:37:06.956974 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:06.956945    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls podName:0b67d17d-e9f0-4b5b-a15f-760f130cae93 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:07.456922893 +0000 UTC m=+36.953228984 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-lntx2" (UID: "0b67d17d-e9f0-4b5b-a15f-760f130cae93") : secret "cluster-monitoring-operator-tls" not found
Sep 19 10:37:06.956974 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:06.956971    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 10:37:06.957323 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957032    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7dg2\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-kube-api-access-g7dg2\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:06.957323 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957141    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2bfbbce9-4493-4ec5-b0d5-d85da7e26e65-serving-cert\") pod \"insights-operator-6f9c644dd9-44pw4\" (UID: \"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65\") " pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:06.957323 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957173    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/26cdce24-be14-4ec9-a4cd-403bde09dd5d-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-96xsm\" (UID: \"26cdce24-be14-4ec9-a4cd-403bde09dd5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-96xsm"
Sep 19 10:37:06.957323 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957215    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrssp\" (UniqueName: \"kubernetes.io/projected/0b67d17d-e9f0-4b5b-a15f-760f130cae93-kube-api-access-xrssp\") pod \"cluster-monitoring-operator-847b8bd8d7-lntx2\" (UID: \"0b67d17d-e9f0-4b5b-a15f-760f130cae93\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"
Sep 19 10:37:06.957323 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957240    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39728838-de41-40c4-8d66-362ab1882457-registry-certificates\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:06.957323 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957282    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-bound-sa-token\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:06.957323 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957310    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbwp4\" (UniqueName: \"kubernetes.io/projected/26cdce24-be14-4ec9-a4cd-403bde09dd5d-kube-api-access-kbwp4\") pod \"kube-storage-version-migrator-operator-55889f7b47-96xsm\" (UID: \"26cdce24-be14-4ec9-a4cd-403bde09dd5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-96xsm"
Sep 19 10:37:06.957648 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-stats-auth\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:06.957648 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957353    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bfbbce9-4493-4ec5-b0d5-d85da7e26e65-tmp\") pod \"insights-operator-6f9c644dd9-44pw4\" (UID: \"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65\") " pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:06.957648 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957365    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dcf226b4-3c7f-40c0-a327-96003c35e314-serving-cert\") pod \"service-ca-operator-57cdc4b768-6dllh\" (UID: \"dcf226b4-3c7f-40c0-a327-96003c35e314\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6dllh"
Sep 19 10:37:06.957648 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:06.957393    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls podName:3755924c-2290-44b0-ad8f-76c6c14c2b27 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:07.457374175 +0000 UTC m=+36.953680251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-qzr7f" (UID: "3755924c-2290-44b0-ad8f-76c6c14c2b27") : secret "samples-operator-tls" not found
Sep 19 10:37:06.957648 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957530    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bfbbce9-4493-4ec5-b0d5-d85da7e26e65-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-44pw4\" (UID: \"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65\") " pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:06.957648 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957637    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dcf226b4-3c7f-40c0-a327-96003c35e314-config\") pod \"service-ca-operator-57cdc4b768-6dllh\" (UID: \"dcf226b4-3c7f-40c0-a327-96003c35e314\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6dllh"
Sep 19 10:37:06.957943 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957733    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0b67d17d-e9f0-4b5b-a15f-760f130cae93-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-lntx2\" (UID: \"0b67d17d-e9f0-4b5b-a15f-760f130cae93\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"
Sep 19 10:37:06.957943 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957769    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmpg9\" (UniqueName: \"kubernetes.io/projected/dcf226b4-3c7f-40c0-a327-96003c35e314-kube-api-access-kmpg9\") pod \"service-ca-operator-57cdc4b768-6dllh\" (UID: \"dcf226b4-3c7f-40c0-a327-96003c35e314\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6dllh"
Sep 19 10:37:06.957943 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957797    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2bfbbce9-4493-4ec5-b0d5-d85da7e26e65-snapshots\") pod \"insights-operator-6f9c644dd9-44pw4\" (UID: \"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65\") " pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:06.957943 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957824    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39728838-de41-40c4-8d66-362ab1882457-image-registry-private-configuration\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:06.957943 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957850    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:06.957943 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957878    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/26cdce24-be14-4ec9-a4cd-403bde09dd5d-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-96xsm\" (UID: \"26cdce24-be14-4ec9-a4cd-403bde09dd5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-96xsm"
Sep 19 10:37:06.957943 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957908    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39728838-de41-40c4-8d66-362ab1882457-ca-trust-extracted\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:06.957943 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957939    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:06.958355 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.957966    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f46ws\" (UniqueName: \"kubernetes.io/projected/18495139-9a71-4bd7-b279-1d7e6470271f-kube-api-access-f46ws\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:06.958355 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.958007    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgpvf\" (UniqueName: \"kubernetes.io/projected/2bfbbce9-4493-4ec5-b0d5-d85da7e26e65-kube-api-access-xgpvf\") pod \"insights-operator-6f9c644dd9-44pw4\" (UID: \"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65\") " pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:06.958355 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.958068    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmnm8\" (UniqueName: \"kubernetes.io/projected/3755924c-2290-44b0-ad8f-76c6c14c2b27-kube-api-access-nmnm8\") pod \"cluster-samples-operator-685d7d68b5-qzr7f\" (UID: \"3755924c-2290-44b0-ad8f-76c6c14c2b27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f"
Sep 19 10:37:06.958355 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.958094    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39728838-de41-40c4-8d66-362ab1882457-installation-pull-secrets\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:06.958355 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.958127    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:06.958355 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.958159    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-default-certificate\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:06.958355 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.958188    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bfbbce9-4493-4ec5-b0d5-d85da7e26e65-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-44pw4\" (UID: \"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65\") " pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:06.958677 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.958568    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2bfbbce9-4493-4ec5-b0d5-d85da7e26e65-snapshots\") pod \"insights-operator-6f9c644dd9-44pw4\" (UID: \"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65\") " pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:06.959269 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.959065    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0b67d17d-e9f0-4b5b-a15f-760f130cae93-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-lntx2\" (UID: \"0b67d17d-e9f0-4b5b-a15f-760f130cae93\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"
Sep 19 10:37:06.959269 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.959122    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bfbbce9-4493-4ec5-b0d5-d85da7e26e65-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-44pw4\" (UID: \"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65\") " pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:06.959269 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:06.959172    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 19 10:37:06.959269 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:06.959218    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle podName:18495139-9a71-4bd7-b279-1d7e6470271f nodeName:}" failed. No retries permitted until 2026-09-19 10:37:07.459200358 +0000 UTC m=+36.955506450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle") pod "router-default-6ddf8bf8dd-66fk8" (UID: "18495139-9a71-4bd7-b279-1d7e6470271f") : configmap references non-existent config key: service-ca.crt
Sep 19 10:37:06.959269 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:06.959257    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs podName:18495139-9a71-4bd7-b279-1d7e6470271f nodeName:}" failed. No retries permitted until 2026-09-19 10:37:07.459240648 +0000 UTC m=+36.955546736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs") pod "router-default-6ddf8bf8dd-66fk8" (UID: "18495139-9a71-4bd7-b279-1d7e6470271f") : secret "router-metrics-certs-default" not found
Sep 19 10:37:06.960822 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.960797    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-stats-auth\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:06.960921 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.960800    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dcf226b4-3c7f-40c0-a327-96003c35e314-serving-cert\") pod \"service-ca-operator-57cdc4b768-6dllh\" (UID: \"dcf226b4-3c7f-40c0-a327-96003c35e314\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6dllh"
Sep 19 10:37:06.960921 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.960866    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2bfbbce9-4493-4ec5-b0d5-d85da7e26e65-serving-cert\") pod \"insights-operator-6f9c644dd9-44pw4\" (UID: \"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65\") " pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:06.961240 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.961223    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-default-certificate\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:06.968242 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.968219    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5nqg9"
Sep 19 10:37:06.968751 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.968722    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrssp\" (UniqueName: \"kubernetes.io/projected/0b67d17d-e9f0-4b5b-a15f-760f130cae93-kube-api-access-xrssp\") pod \"cluster-monitoring-operator-847b8bd8d7-lntx2\" (UID: \"0b67d17d-e9f0-4b5b-a15f-760f130cae93\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"
Sep 19 10:37:06.968867 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.968848    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmnm8\" (UniqueName: \"kubernetes.io/projected/3755924c-2290-44b0-ad8f-76c6c14c2b27-kube-api-access-nmnm8\") pod \"cluster-samples-operator-685d7d68b5-qzr7f\" (UID: \"3755924c-2290-44b0-ad8f-76c6c14c2b27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f"
Sep 19 10:37:06.968993 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.968868    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmpg9\" (UniqueName: \"kubernetes.io/projected/dcf226b4-3c7f-40c0-a327-96003c35e314-kube-api-access-kmpg9\") pod \"service-ca-operator-57cdc4b768-6dllh\" (UID: \"dcf226b4-3c7f-40c0-a327-96003c35e314\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6dllh"
Sep 19 10:37:06.968993 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.968951    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f46ws\" (UniqueName: \"kubernetes.io/projected/18495139-9a71-4bd7-b279-1d7e6470271f-kube-api-access-f46ws\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:06.969798 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.969782    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgpvf\" (UniqueName: \"kubernetes.io/projected/2bfbbce9-4493-4ec5-b0d5-d85da7e26e65-kube-api-access-xgpvf\") pod \"insights-operator-6f9c644dd9-44pw4\" (UID: \"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65\") " pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:06.978715 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:06.978690    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-m25hk"
Sep 19 10:37:07.059145 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.059116    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/26cdce24-be14-4ec9-a4cd-403bde09dd5d-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-96xsm\" (UID: \"26cdce24-be14-4ec9-a4cd-403bde09dd5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-96xsm"
Sep 19 10:37:07.059260 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.059168    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39728838-de41-40c4-8d66-362ab1882457-ca-trust-extracted\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:07.059260 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.059228    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39728838-de41-40c4-8d66-362ab1882457-installation-pull-secrets\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:07.059449 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.059282    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39728838-de41-40c4-8d66-362ab1882457-trusted-ca\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:07.059449 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.059347    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7dg2\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-kube-api-access-g7dg2\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:07.059449 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.059393    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/26cdce24-be14-4ec9-a4cd-403bde09dd5d-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-96xsm\" (UID: \"26cdce24-be14-4ec9-a4cd-403bde09dd5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-96xsm"
Sep 19 10:37:07.059449 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.059432    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39728838-de41-40c4-8d66-362ab1882457-registry-certificates\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:07.059725 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.059468    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls\") pod \"dns-default-wf45t\" (UID: \"daad582d-23b8-445c-8c99-dec7c11c95a7\") " pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:07.059725 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.059508    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-bound-sa-token\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:07.059725 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.059551    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbwp4\" (UniqueName: \"kubernetes.io/projected/26cdce24-be14-4ec9-a4cd-403bde09dd5d-kube-api-access-kbwp4\") pod \"kube-storage-version-migrator-operator-55889f7b47-96xsm\" (UID: \"26cdce24-be14-4ec9-a4cd-403bde09dd5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-96xsm"
Sep 19 10:37:07.065261 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:07.060853    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 10:37:07.065261 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.060916    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39728838-de41-40c4-8d66-362ab1882457-registry-certificates\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:07.065261 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.060921    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/26cdce24-be14-4ec9-a4cd-403bde09dd5d-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-96xsm\" (UID: \"26cdce24-be14-4ec9-a4cd-403bde09dd5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-96xsm"
Sep 19 10:37:07.065261 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:07.060927    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls podName:daad582d-23b8-445c-8c99-dec7c11c95a7 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:09.060905072 +0000 UTC m=+38.557211162 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls") pod "dns-default-wf45t" (UID: "daad582d-23b8-445c-8c99-dec7c11c95a7") : secret "dns-default-metrics-tls" not found
Sep 19 10:37:07.065261 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.061594    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39728838-de41-40c4-8d66-362ab1882457-ca-trust-extracted\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:07.065261 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.061614    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39728838-de41-40c4-8d66-362ab1882457-image-registry-private-configuration\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:07.065261 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.061647    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:07.065261 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:07.061766    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 10:37:07.065261 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:07.061779    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77fdd969cd-nq42r: secret "image-registry-tls" not found
Sep 19 10:37:07.065261 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:07.061829    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls podName:39728838-de41-40c4-8d66-362ab1882457 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:07.561811846 +0000 UTC m=+37.058117920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls") pod "image-registry-77fdd969cd-nq42r" (UID: "39728838-de41-40c4-8d66-362ab1882457") : secret "image-registry-tls" not found
Sep 19 10:37:07.065261 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.062212    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39728838-de41-40c4-8d66-362ab1882457-trusted-ca\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:07.065261 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.064320    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/26cdce24-be14-4ec9-a4cd-403bde09dd5d-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-96xsm\" (UID: \"26cdce24-be14-4ec9-a4cd-403bde09dd5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-96xsm"
Sep 19 10:37:07.065261 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.064385    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39728838-de41-40c4-8d66-362ab1882457-installation-pull-secrets\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:07.065261 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.064717    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39728838-de41-40c4-8d66-362ab1882457-image-registry-private-configuration\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:07.069567 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.069130    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6dllh"
Sep 19 10:37:07.072941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.072906    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7dg2\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-kube-api-access-g7dg2\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:07.073402 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.073384    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbwp4\" (UniqueName: \"kubernetes.io/projected/26cdce24-be14-4ec9-a4cd-403bde09dd5d-kube-api-access-kbwp4\") pod \"kube-storage-version-migrator-operator-55889f7b47-96xsm\" (UID: \"26cdce24-be14-4ec9-a4cd-403bde09dd5d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-96xsm"
Sep 19 10:37:07.074520 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.074496    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-bound-sa-token\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:07.102702 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.102674    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-44pw4"
Sep 19 10:37:07.190554 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.190437    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-96xsm"
Sep 19 10:37:07.204575 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.204545    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g5qhx"]
Sep 19 10:37:07.253510 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.253480    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5nqg9"]
Sep 19 10:37:07.276936 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.276891    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-m25hk"]
Sep 19 10:37:07.300420 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.300390    2576 generic.go:356] "Generic (PLEG): container finished" podID="e10a7fca-fab2-40fe-888c-425f875e9a3e" containerID="16508185d260f05b65c4e1cd76dd7e4b7efb124ac314368ab88d109ddd72b905" exitCode=0
Sep 19 10:37:07.300572 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.300454    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z6nh5" event={"ID":"e10a7fca-fab2-40fe-888c-425f875e9a3e","Type":"ContainerDied","Data":"16508185d260f05b65c4e1cd76dd7e4b7efb124ac314368ab88d109ddd72b905"}
Sep 19 10:37:07.353011 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.352884    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-6dllh"]
Sep 19 10:37:07.378092 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.378036    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-44pw4"]
Sep 19 10:37:07.382310 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:37:07.382267    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod320e460b_f15a_4da5_82eb_f92a0f756180.slice/crio-4d96f6304feb96d417b7a9aa8e67580af05bc305fe91b7292378b48f9465aeab WatchSource:0}: Error finding container 4d96f6304feb96d417b7a9aa8e67580af05bc305fe91b7292378b48f9465aeab: Status 404 returned error can't find the container with id 4d96f6304feb96d417b7a9aa8e67580af05bc305fe91b7292378b48f9465aeab
Sep 19 10:37:07.383059 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:37:07.383006    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd526e598_cec6_460b_abcc_e5bee3890d8e.slice/crio-331bb7b6f79795e6c2638643f38f66ea701eb45943faa53309cc9fbdcbaa353e WatchSource:0}: Error finding container 331bb7b6f79795e6c2638643f38f66ea701eb45943faa53309cc9fbdcbaa353e: Status 404 returned error can't find the container with id 331bb7b6f79795e6c2638643f38f66ea701eb45943faa53309cc9fbdcbaa353e
Sep 19 10:37:07.465375 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.465285    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:07.465375 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.465334    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-lntx2\" (UID: \"0b67d17d-e9f0-4b5b-a15f-760f130cae93\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"
Sep 19 10:37:07.465610 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.465380    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-qzr7f\" (UID: \"3755924c-2290-44b0-ad8f-76c6c14c2b27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f"
Sep 19 10:37:07.465610 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:07.465461    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle podName:18495139-9a71-4bd7-b279-1d7e6470271f nodeName:}" failed. No retries permitted until 2026-09-19 10:37:08.465434114 +0000 UTC m=+37.961740198 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle") pod "router-default-6ddf8bf8dd-66fk8" (UID: "18495139-9a71-4bd7-b279-1d7e6470271f") : configmap references non-existent config key: service-ca.crt
Sep 19 10:37:07.465610 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:07.465502    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 10:37:07.465610 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:07.465521    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 10:37:07.465610 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:07.465555    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls podName:0b67d17d-e9f0-4b5b-a15f-760f130cae93 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:08.465542951 +0000 UTC m=+37.961849024 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-lntx2" (UID: "0b67d17d-e9f0-4b5b-a15f-760f130cae93") : secret "cluster-monitoring-operator-tls" not found
Sep 19 10:37:07.465610 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:07.465580    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls podName:3755924c-2290-44b0-ad8f-76c6c14c2b27 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:08.465562893 +0000 UTC m=+37.961868983 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-qzr7f" (UID: "3755924c-2290-44b0-ad8f-76c6c14c2b27") : secret "samples-operator-tls" not found
Sep 19 10:37:07.465912 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.465696    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:07.465912 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:07.465820    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 19 10:37:07.465912 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:07.465867    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs podName:18495139-9a71-4bd7-b279-1d7e6470271f nodeName:}" failed. No retries permitted until 2026-09-19 10:37:08.465854851 +0000 UTC m=+37.962160941 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs") pod "router-default-6ddf8bf8dd-66fk8" (UID: "18495139-9a71-4bd7-b279-1d7e6470271f") : secret "router-metrics-certs-default" not found
Sep 19 10:37:07.566745 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.566715    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:07.566896 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:07.566832    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 10:37:07.566896 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:07.566843    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77fdd969cd-nq42r: secret "image-registry-tls" not found
Sep 19 10:37:07.566961 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:07.566900    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls podName:39728838-de41-40c4-8d66-362ab1882457 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:08.566885446 +0000 UTC m=+38.063191516 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls") pod "image-registry-77fdd969cd-nq42r" (UID: "39728838-de41-40c4-8d66-362ab1882457") : secret "image-registry-tls" not found
Sep 19 10:37:07.646678 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.646610    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-96xsm"]
Sep 19 10:37:07.650561 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:37:07.650529    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26cdce24_be14_4ec9_a4cd_403bde09dd5d.slice/crio-b96c0c8206f7998ab5496b44aba1fb6a8e82ed351e3fa488c98f4c7f6f197e88 WatchSource:0}: Error finding container b96c0c8206f7998ab5496b44aba1fb6a8e82ed351e3fa488c98f4c7f6f197e88: Status 404 returned error can't find the container with id b96c0c8206f7998ab5496b44aba1fb6a8e82ed351e3fa488c98f4c7f6f197e88
Sep 19 10:37:07.966837 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.966796    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-rlwbq"]
Sep 19 10:37:07.982218 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.981503    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-rlwbq"]
Sep 19 10:37:07.982218 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.981670    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-rlwbq"
Sep 19 10:37:07.985742 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:07.984969    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-sqpkr\""
Sep 19 10:37:08.070887 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:08.070848    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69k7p\" (UniqueName: \"kubernetes.io/projected/e3fa884e-f0e3-401e-b4a6-d0f3cd7a6f5d-kube-api-access-69k7p\") pod \"network-check-source-7cbc45fddc-rlwbq\" (UID: \"e3fa884e-f0e3-401e-b4a6-d0f3cd7a6f5d\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-rlwbq"
Sep 19 10:37:08.172386 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:08.172344    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69k7p\" (UniqueName: \"kubernetes.io/projected/e3fa884e-f0e3-401e-b4a6-d0f3cd7a6f5d-kube-api-access-69k7p\") pod \"network-check-source-7cbc45fddc-rlwbq\" (UID: \"e3fa884e-f0e3-401e-b4a6-d0f3cd7a6f5d\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-rlwbq"
Sep 19 10:37:08.183262 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:08.183134    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69k7p\" (UniqueName: \"kubernetes.io/projected/e3fa884e-f0e3-401e-b4a6-d0f3cd7a6f5d-kube-api-access-69k7p\") pod \"network-check-source-7cbc45fddc-rlwbq\" (UID: \"e3fa884e-f0e3-401e-b4a6-d0f3cd7a6f5d\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-rlwbq"
Sep 19 10:37:08.298458 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:08.298068    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-rlwbq"
Sep 19 10:37:08.314179 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:08.314074    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-96xsm" event={"ID":"26cdce24-be14-4ec9-a4cd-403bde09dd5d","Type":"ContainerStarted","Data":"b96c0c8206f7998ab5496b44aba1fb6a8e82ed351e3fa488c98f4c7f6f197e88"}
Sep 19 10:37:08.315941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:08.315850    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-44pw4" event={"ID":"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65","Type":"ContainerStarted","Data":"31f55522db7cb8b0a352466e718fb83ebd0fb108dde305152a1d029d61c72d1d"}
Sep 19 10:37:08.321939 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:08.321861    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6dllh" event={"ID":"dcf226b4-3c7f-40c0-a327-96003c35e314","Type":"ContainerStarted","Data":"386b0b9d94693a2c23518273be37edee19568b44f17158979304ffdc0b0c6875"}
Sep 19 10:37:08.330148 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:08.329574    2576 generic.go:356] "Generic (PLEG): container finished" podID="e10a7fca-fab2-40fe-888c-425f875e9a3e" containerID="dfbf8ded9d0315f9b7a4d1f2abd294fae434e5ac7763b256241d16cf2887ecc1" exitCode=0
Sep 19 10:37:08.330148 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:08.329785    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z6nh5" event={"ID":"e10a7fca-fab2-40fe-888c-425f875e9a3e","Type":"ContainerDied","Data":"dfbf8ded9d0315f9b7a4d1f2abd294fae434e5ac7763b256241d16cf2887ecc1"}
Sep 19 10:37:08.331746 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:08.331690    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-m25hk" event={"ID":"320e460b-f15a-4da5-82eb-f92a0f756180","Type":"ContainerStarted","Data":"4d96f6304feb96d417b7a9aa8e67580af05bc305fe91b7292378b48f9465aeab"}
Sep 19 10:37:08.334526 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:08.334504    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5nqg9" event={"ID":"d526e598-cec6-460b-abcc-e5bee3890d8e","Type":"ContainerStarted","Data":"331bb7b6f79795e6c2638643f38f66ea701eb45943faa53309cc9fbdcbaa353e"}
Sep 19 10:37:08.336493 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:08.336441    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g5qhx" event={"ID":"bd12ed6b-5433-4b19-9eba-1973e0bd4afa","Type":"ContainerStarted","Data":"73bd9c72bfdf983d1148b7b25bb4f712114a56a163b7923acba0db2caf737490"}
Sep 19 10:37:08.475199 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:08.475158    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:08.475411 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:08.475231    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:08.475411 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:08.475280    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-lntx2\" (UID: \"0b67d17d-e9f0-4b5b-a15f-760f130cae93\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"
Sep 19 10:37:08.475411 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:08.475312    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-qzr7f\" (UID: \"3755924c-2290-44b0-ad8f-76c6c14c2b27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f"
Sep 19 10:37:08.476726 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:08.475729    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle podName:18495139-9a71-4bd7-b279-1d7e6470271f nodeName:}" failed. No retries permitted until 2026-09-19 10:37:10.475707028 +0000 UTC m=+39.972013123 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle") pod "router-default-6ddf8bf8dd-66fk8" (UID: "18495139-9a71-4bd7-b279-1d7e6470271f") : configmap references non-existent config key: service-ca.crt
Sep 19 10:37:08.476726 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:08.475832    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 10:37:08.476726 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:08.475903    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls podName:3755924c-2290-44b0-ad8f-76c6c14c2b27 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:10.475883781 +0000 UTC m=+39.972189871 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-qzr7f" (UID: "3755924c-2290-44b0-ad8f-76c6c14c2b27") : secret "samples-operator-tls" not found
Sep 19 10:37:08.476726 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:08.475975    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 10:37:08.476726 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:08.476012    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls podName:0b67d17d-e9f0-4b5b-a15f-760f130cae93 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:10.476000212 +0000 UTC m=+39.972306309 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-lntx2" (UID: "0b67d17d-e9f0-4b5b-a15f-760f130cae93") : secret "cluster-monitoring-operator-tls" not found
Sep 19 10:37:08.476726 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:08.476419    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 19 10:37:08.476726 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:08.476458    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs podName:18495139-9a71-4bd7-b279-1d7e6470271f nodeName:}" failed. No retries permitted until 2026-09-19 10:37:10.476445914 +0000 UTC m=+39.972751990 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs") pod "router-default-6ddf8bf8dd-66fk8" (UID: "18495139-9a71-4bd7-b279-1d7e6470271f") : secret "router-metrics-certs-default" not found
Sep 19 10:37:08.576611 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:08.576382    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:08.576868 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:08.576846    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 10:37:08.576951 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:08.576871    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77fdd969cd-nq42r: secret "image-registry-tls" not found
Sep 19 10:37:08.576951 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:08.576934    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls podName:39728838-de41-40c4-8d66-362ab1882457 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:10.576914179 +0000 UTC m=+40.073220268 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls") pod "image-registry-77fdd969cd-nq42r" (UID: "39728838-de41-40c4-8d66-362ab1882457") : secret "image-registry-tls" not found
Sep 19 10:37:08.608886 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:08.608845    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-rlwbq"]
Sep 19 10:37:08.615180 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:37:08.615139    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3fa884e_f0e3_401e_b4a6_d0f3cd7a6f5d.slice/crio-d1144c9939063e78cb58a8ad7a052abba9ed7a8432a10ba8182e13881f01d1ea WatchSource:0}: Error finding container d1144c9939063e78cb58a8ad7a052abba9ed7a8432a10ba8182e13881f01d1ea: Status 404 returned error can't find the container with id d1144c9939063e78cb58a8ad7a052abba9ed7a8432a10ba8182e13881f01d1ea
Sep 19 10:37:09.081647 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:09.081589    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls\") pod \"dns-default-wf45t\" (UID: \"daad582d-23b8-445c-8c99-dec7c11c95a7\") " pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:09.082173 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:09.081848    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 10:37:09.082173 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:09.081912    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls podName:daad582d-23b8-445c-8c99-dec7c11c95a7 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:13.081891937 +0000 UTC m=+42.578198010 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls") pod "dns-default-wf45t" (UID: "daad582d-23b8-445c-8c99-dec7c11c95a7") : secret "dns-default-metrics-tls" not found
Sep 19 10:37:09.356072 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:09.354264    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z6nh5" event={"ID":"e10a7fca-fab2-40fe-888c-425f875e9a3e","Type":"ContainerStarted","Data":"88640454be5383747d654bc35fc4e4c24158c38e4c717757c8397a5518ab79ae"}
Sep 19 10:37:09.360971 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:09.360934    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-rlwbq" event={"ID":"e3fa884e-f0e3-401e-b4a6-d0f3cd7a6f5d","Type":"ContainerStarted","Data":"d1144c9939063e78cb58a8ad7a052abba9ed7a8432a10ba8182e13881f01d1ea"}
Sep 19 10:37:09.377125 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:09.376174    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-z6nh5" podStartSLOduration=8.051160407 podStartE2EDuration="38.376153152s" totalImagesPullingTime="30.324992745s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:36:31 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:37:09.374816182 +0000 UTC m=+38.871122298" watchObservedRunningTime="2026-09-19 10:37:09.376153152 +0000 UTC m=+38.872459245"
Sep 19 10:37:10.366846 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:10.366743    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5nqg9" event={"ID":"d526e598-cec6-460b-abcc-e5bee3890d8e","Type":"ContainerStarted","Data":"ee66234ec2caa4f1e8c5236072e1d757354635c19add7389b0572e63cbef0851"}
Sep 19 10:37:10.382156 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:10.382092    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5nqg9" podStartSLOduration=2.336593289 podStartE2EDuration="4.382067221s" totalImagesPullingTime="2.045473932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:37:10.380615392 +0000 UTC m=+39.876921486" watchObservedRunningTime="2026-09-19 10:37:10.382067221 +0000 UTC m=+39.878373309"
Sep 19 10:37:10.496433 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:10.496391    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-lntx2\" (UID: \"0b67d17d-e9f0-4b5b-a15f-760f130cae93\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"
Sep 19 10:37:10.496582 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:10.496444    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-qzr7f\" (UID: \"3755924c-2290-44b0-ad8f-76c6c14c2b27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f"
Sep 19 10:37:10.496641 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:10.496586    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:10.496641 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:10.496629    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:10.496785 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:10.496772    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle podName:18495139-9a71-4bd7-b279-1d7e6470271f nodeName:}" failed. No retries permitted until 2026-09-19 10:37:14.496758026 +0000 UTC m=+43.993064096 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle") pod "router-default-6ddf8bf8dd-66fk8" (UID: "18495139-9a71-4bd7-b279-1d7e6470271f") : configmap references non-existent config key: service-ca.crt
Sep 19 10:37:10.497177 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:10.497159    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 10:37:10.497232 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:10.497213    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls podName:0b67d17d-e9f0-4b5b-a15f-760f130cae93 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:14.497198435 +0000 UTC m=+43.993504505 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-lntx2" (UID: "0b67d17d-e9f0-4b5b-a15f-760f130cae93") : secret "cluster-monitoring-operator-tls" not found
Sep 19 10:37:10.497299 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:10.497280    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 10:37:10.497374 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:10.497316    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls podName:3755924c-2290-44b0-ad8f-76c6c14c2b27 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:14.497305919 +0000 UTC m=+43.993611989 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-qzr7f" (UID: "3755924c-2290-44b0-ad8f-76c6c14c2b27") : secret "samples-operator-tls" not found
Sep 19 10:37:10.497606 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:10.497581    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 19 10:37:10.497707 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:10.497647    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs podName:18495139-9a71-4bd7-b279-1d7e6470271f nodeName:}" failed. No retries permitted until 2026-09-19 10:37:14.497629387 +0000 UTC m=+43.993935466 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs") pod "router-default-6ddf8bf8dd-66fk8" (UID: "18495139-9a71-4bd7-b279-1d7e6470271f") : secret "router-metrics-certs-default" not found
Sep 19 10:37:10.597440 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:10.597395    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:10.597629 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:10.597553    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 10:37:10.597629 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:10.597577    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77fdd969cd-nq42r: secret "image-registry-tls" not found
Sep 19 10:37:10.597751 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:10.597651    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls podName:39728838-de41-40c4-8d66-362ab1882457 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:14.597626815 +0000 UTC m=+44.093932934 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls") pod "image-registry-77fdd969cd-nq42r" (UID: "39728838-de41-40c4-8d66-362ab1882457") : secret "image-registry-tls" not found
Sep 19 10:37:13.123296 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:13.123250    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls\") pod \"dns-default-wf45t\" (UID: \"daad582d-23b8-445c-8c99-dec7c11c95a7\") " pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:13.123748 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:13.123410    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 10:37:13.123748 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:13.123500    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls podName:daad582d-23b8-445c-8c99-dec7c11c95a7 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:21.123480557 +0000 UTC m=+50.619786647 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls") pod "dns-default-wf45t" (UID: "daad582d-23b8-445c-8c99-dec7c11c95a7") : secret "dns-default-metrics-tls" not found
Sep 19 10:37:14.538007 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:14.537959    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:14.538451 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:14.538057    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-lntx2\" (UID: \"0b67d17d-e9f0-4b5b-a15f-760f130cae93\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"
Sep 19 10:37:14.538451 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:14.538096    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-qzr7f\" (UID: \"3755924c-2290-44b0-ad8f-76c6c14c2b27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f"
Sep 19 10:37:14.538451 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:14.538185    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle podName:18495139-9a71-4bd7-b279-1d7e6470271f nodeName:}" failed. No retries permitted until 2026-09-19 10:37:22.538159698 +0000 UTC m=+52.034465788 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle") pod "router-default-6ddf8bf8dd-66fk8" (UID: "18495139-9a71-4bd7-b279-1d7e6470271f") : configmap references non-existent config key: service-ca.crt
Sep 19 10:37:14.538451 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:14.538235    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 10:37:14.538451 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:14.538237    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 10:37:14.538451 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:14.538281    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls podName:3755924c-2290-44b0-ad8f-76c6c14c2b27 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:22.538269195 +0000 UTC m=+52.034575264 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-qzr7f" (UID: "3755924c-2290-44b0-ad8f-76c6c14c2b27") : secret "samples-operator-tls" not found
Sep 19 10:37:14.538451 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:14.538298    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls podName:0b67d17d-e9f0-4b5b-a15f-760f130cae93 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:22.538291999 +0000 UTC m=+52.034598068 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-lntx2" (UID: "0b67d17d-e9f0-4b5b-a15f-760f130cae93") : secret "cluster-monitoring-operator-tls" not found
Sep 19 10:37:14.538451 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:14.538317    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:14.538451 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:14.538390    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 19 10:37:14.538451 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:14.538416    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs podName:18495139-9a71-4bd7-b279-1d7e6470271f nodeName:}" failed. No retries permitted until 2026-09-19 10:37:22.538409542 +0000 UTC m=+52.034715611 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs") pod "router-default-6ddf8bf8dd-66fk8" (UID: "18495139-9a71-4bd7-b279-1d7e6470271f") : secret "router-metrics-certs-default" not found
Sep 19 10:37:14.638796 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:14.638750    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:14.638966 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:14.638921    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 10:37:14.638966 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:14.638942    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77fdd969cd-nq42r: secret "image-registry-tls" not found
Sep 19 10:37:14.639057 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:14.639001    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls podName:39728838-de41-40c4-8d66-362ab1882457 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:22.638983257 +0000 UTC m=+52.135289332 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls") pod "image-registry-77fdd969cd-nq42r" (UID: "39728838-de41-40c4-8d66-362ab1882457") : secret "image-registry-tls" not found
Sep 19 10:37:15.382604 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.382570    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-96xsm" event={"ID":"26cdce24-be14-4ec9-a4cd-403bde09dd5d","Type":"ContainerStarted","Data":"00e4d297de1c6649a2a6d735a0efcf44302c6e37c1ce170fb5ba86990c583a62"}
Sep 19 10:37:15.384445 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.384415    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-44pw4" event={"ID":"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65","Type":"ContainerStarted","Data":"e4125f40b421490e7b11848c882301714af1a43066b073eb26df04eddf95ce75"}
Sep 19 10:37:15.394296 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.394264    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6dllh" event={"ID":"dcf226b4-3c7f-40c0-a327-96003c35e314","Type":"ContainerStarted","Data":"3759b99aecd166bdc282ab56043ba1f1715da8622dcc4767f09cbd7a3f79fd10"}
Sep 19 10:37:15.396280 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.396258    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/0.log"
Sep 19 10:37:15.396409 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.396299    2576 generic.go:356] "Generic (PLEG): container finished" podID="320e460b-f15a-4da5-82eb-f92a0f756180" containerID="762d3bd163632e8e24ccb780aa3ac1616b5262f7b72a1cd309d8d7b2e283ad33" exitCode=255
Sep 19 10:37:15.396409 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.396381    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-m25hk" event={"ID":"320e460b-f15a-4da5-82eb-f92a0f756180","Type":"ContainerDied","Data":"762d3bd163632e8e24ccb780aa3ac1616b5262f7b72a1cd309d8d7b2e283ad33"}
Sep 19 10:37:15.396555 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.396542    2576 scope.go:117] "RemoveContainer" containerID="762d3bd163632e8e24ccb780aa3ac1616b5262f7b72a1cd309d8d7b2e283ad33"
Sep 19 10:37:15.399015 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.398651    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g5qhx" event={"ID":"bd12ed6b-5433-4b19-9eba-1973e0bd4afa","Type":"ContainerStarted","Data":"e8210ad8d5cf89edee0dab72ac47b17fbdd29cd788abf61fa25412c14ae5fbc3"}
Sep 19 10:37:15.400221 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.400198    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-rlwbq" event={"ID":"e3fa884e-f0e3-401e-b4a6-d0f3cd7a6f5d","Type":"ContainerStarted","Data":"d2450629f76c4647d290b8cf968ac52095d576b5fd38e84bf046b8b5d9345f6f"}
Sep 19 10:37:15.404959 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.404912    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-96xsm" podStartSLOduration=2.0803507 podStartE2EDuration="9.404898276s" totalImagesPullingTime="7.324547576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:37:15.403637112 +0000 UTC m=+44.899943206" watchObservedRunningTime="2026-09-19 10:37:15.404898276 +0000 UTC m=+44.901204407"
Sep 19 10:37:15.436966 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.436907    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-6f9c644dd9-44pw4" podStartSLOduration=1.8675040429999998 podStartE2EDuration="9.436885784s" totalImagesPullingTime="7.569381741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:37:15.436110375 +0000 UTC m=+44.932416465" watchObservedRunningTime="2026-09-19 10:37:15.436885784 +0000 UTC m=+44.933191877"
Sep 19 10:37:15.452663 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.452606    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-rlwbq" podStartSLOduration=2.083770019 podStartE2EDuration="8.452583122s" totalImagesPullingTime="6.368813103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:37:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:37:15.451918078 +0000 UTC m=+44.948224173" watchObservedRunningTime="2026-09-19 10:37:15.452583122 +0000 UTC m=+44.948889214"
Sep 19 10:37:15.475279 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.471300    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6dllh" podStartSLOduration=1.88311937 podStartE2EDuration="9.471265538s" totalImagesPullingTime="7.588146168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:37:15.467329494 +0000 UTC m=+44.963635587" watchObservedRunningTime="2026-09-19 10:37:15.471265538 +0000 UTC m=+44.967571633"
Sep 19 10:37:15.483939 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.483836    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-g5qhx" podStartSLOduration=2.895824853 podStartE2EDuration="10.483811576s" totalImagesPullingTime="7.587986723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:37:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:37:15.483804717 +0000 UTC m=+44.980110809" watchObservedRunningTime="2026-09-19 10:37:15.483811576 +0000 UTC m=+44.980117669"
Sep 19 10:37:15.787783 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.787701    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lch4t"]
Sep 19 10:37:15.806252 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.806223    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lch4t"]
Sep 19 10:37:15.806441 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.806370    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:15.809008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.808822    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 19 10:37:15.809008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.808837    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5c6t4\""
Sep 19 10:37:15.809236 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.809182    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 19 10:37:15.850661 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.850628    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d745z\" (UniqueName: \"kubernetes.io/projected/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-api-access-d745z\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:15.850943 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.850925    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2de2ff6b-99b1-4a41-ba5e-a30034617fac-crio-socket\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:15.851091 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.851076    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2de2ff6b-99b1-4a41-ba5e-a30034617fac-data-volume\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:15.851229 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.851214    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:15.851407 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.851384    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:15.952549 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.952504    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d745z\" (UniqueName: \"kubernetes.io/projected/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-api-access-d745z\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:15.952762 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.952598    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2de2ff6b-99b1-4a41-ba5e-a30034617fac-crio-socket\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:15.952762 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.952626    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2de2ff6b-99b1-4a41-ba5e-a30034617fac-data-volume\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:15.952762 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.952664    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:15.952762 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.952735    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:15.952762 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.952737    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2de2ff6b-99b1-4a41-ba5e-a30034617fac-crio-socket\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:15.953064 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:15.952833    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 10:37:15.953064 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:15.952864    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 10:37:15.953064 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:15.952894    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-rbac-proxy-cm podName:2de2ff6b-99b1-4a41-ba5e-a30034617fac nodeName:}" failed. No retries permitted until 2026-09-19 10:37:16.452874003 +0000 UTC m=+45.949180080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lch4t" (UID: "2de2ff6b-99b1-4a41-ba5e-a30034617fac") : configmap "kube-rbac-proxy" not found
Sep 19 10:37:15.953064 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:15.952932    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls podName:2de2ff6b-99b1-4a41-ba5e-a30034617fac nodeName:}" failed. No retries permitted until 2026-09-19 10:37:16.45291154 +0000 UTC m=+45.949217628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lch4t" (UID: "2de2ff6b-99b1-4a41-ba5e-a30034617fac") : secret "insights-runtime-extractor-tls" not found
Sep 19 10:37:15.953064 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.952998    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2de2ff6b-99b1-4a41-ba5e-a30034617fac-data-volume\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:15.965312 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:15.965263    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d745z\" (UniqueName: \"kubernetes.io/projected/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-api-access-d745z\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:16.405175 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:16.405148    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 10:37:16.405637 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:16.405618    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/0.log"
Sep 19 10:37:16.405773 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:16.405661    2576 generic.go:356] "Generic (PLEG): container finished" podID="320e460b-f15a-4da5-82eb-f92a0f756180" containerID="2fe6f844c3faed1f56ca5fef61a847451688670bcebde7eed2c3a479cec18db1" exitCode=255
Sep 19 10:37:16.405818 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:16.405766    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-m25hk" event={"ID":"320e460b-f15a-4da5-82eb-f92a0f756180","Type":"ContainerDied","Data":"2fe6f844c3faed1f56ca5fef61a847451688670bcebde7eed2c3a479cec18db1"}
Sep 19 10:37:16.405853 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:16.405845    2576 scope.go:117] "RemoveContainer" containerID="762d3bd163632e8e24ccb780aa3ac1616b5262f7b72a1cd309d8d7b2e283ad33"
Sep 19 10:37:16.406159 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:16.406135    2576 scope.go:117] "RemoveContainer" containerID="2fe6f844c3faed1f56ca5fef61a847451688670bcebde7eed2c3a479cec18db1"
Sep 19 10:37:16.406355 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:16.406327    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-m25hk_openshift-console-operator(320e460b-f15a-4da5-82eb-f92a0f756180)\"" pod="openshift-console-operator/console-operator-fd4bb46c-m25hk" podUID="320e460b-f15a-4da5-82eb-f92a0f756180"
Sep 19 10:37:16.457243 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:16.457189    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:16.457444 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:16.457295    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 10:37:16.457444 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:16.457368    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls podName:2de2ff6b-99b1-4a41-ba5e-a30034617fac nodeName:}" failed. No retries permitted until 2026-09-19 10:37:17.457347824 +0000 UTC m=+46.953653908 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lch4t" (UID: "2de2ff6b-99b1-4a41-ba5e-a30034617fac") : secret "insights-runtime-extractor-tls" not found
Sep 19 10:37:16.457712 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:16.457690    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:16.457826 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:16.457776    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 10:37:16.457826 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:16.457823    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-rbac-proxy-cm podName:2de2ff6b-99b1-4a41-ba5e-a30034617fac nodeName:}" failed. No retries permitted until 2026-09-19 10:37:17.457806956 +0000 UTC m=+46.954113027 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lch4t" (UID: "2de2ff6b-99b1-4a41-ba5e-a30034617fac") : configmap "kube-rbac-proxy" not found
Sep 19 10:37:16.979108 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:16.979068    2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-m25hk"
Sep 19 10:37:16.979488 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:16.979152    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-m25hk"
Sep 19 10:37:17.411251 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:17.411224    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 10:37:17.411618 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:17.411601    2576 scope.go:117] "RemoveContainer" containerID="2fe6f844c3faed1f56ca5fef61a847451688670bcebde7eed2c3a479cec18db1"
Sep 19 10:37:17.411838 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:17.411818    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-m25hk_openshift-console-operator(320e460b-f15a-4da5-82eb-f92a0f756180)\"" pod="openshift-console-operator/console-operator-fd4bb46c-m25hk" podUID="320e460b-f15a-4da5-82eb-f92a0f756180"
Sep 19 10:37:17.467228 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:17.467191    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:17.467432 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:17.467300    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 10:37:17.467432 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:17.467383    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls podName:2de2ff6b-99b1-4a41-ba5e-a30034617fac nodeName:}" failed. No retries permitted until 2026-09-19 10:37:19.467366957 +0000 UTC m=+48.963673039 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lch4t" (UID: "2de2ff6b-99b1-4a41-ba5e-a30034617fac") : secret "insights-runtime-extractor-tls" not found
Sep 19 10:37:17.467432 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:17.467432    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:17.467670 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:17.467609    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 10:37:17.467726 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:17.467679    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-rbac-proxy-cm podName:2de2ff6b-99b1-4a41-ba5e-a30034617fac nodeName:}" failed. No retries permitted until 2026-09-19 10:37:19.467661628 +0000 UTC m=+48.963967699 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lch4t" (UID: "2de2ff6b-99b1-4a41-ba5e-a30034617fac") : configmap "kube-rbac-proxy" not found
Sep 19 10:37:17.760497 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:17.760469    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5f6pp_e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d/dns-node-resolver/0.log"
Sep 19 10:37:18.414986 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:18.414959    2576 scope.go:117] "RemoveContainer" containerID="2fe6f844c3faed1f56ca5fef61a847451688670bcebde7eed2c3a479cec18db1"
Sep 19 10:37:18.415390 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:18.415153    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-m25hk_openshift-console-operator(320e460b-f15a-4da5-82eb-f92a0f756180)\"" pod="openshift-console-operator/console-operator-fd4bb46c-m25hk" podUID="320e460b-f15a-4da5-82eb-f92a0f756180"
Sep 19 10:37:18.761065 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:18.760970    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jvnt2_3bda56d0-e1c8-42c2-8905-9719bd0eace8/node-ca/0.log"
Sep 19 10:37:19.486823 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:19.486785    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:19.487334 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:19.486850    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:19.487334 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:19.486972    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 10:37:19.487334 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:19.486973    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 10:37:19.487334 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:19.487060    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-rbac-proxy-cm podName:2de2ff6b-99b1-4a41-ba5e-a30034617fac nodeName:}" failed. No retries permitted until 2026-09-19 10:37:23.487023532 +0000 UTC m=+52.983329602 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lch4t" (UID: "2de2ff6b-99b1-4a41-ba5e-a30034617fac") : configmap "kube-rbac-proxy" not found
Sep 19 10:37:19.487334 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:19.487077    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls podName:2de2ff6b-99b1-4a41-ba5e-a30034617fac nodeName:}" failed. No retries permitted until 2026-09-19 10:37:23.487069134 +0000 UTC m=+52.983375203 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lch4t" (UID: "2de2ff6b-99b1-4a41-ba5e-a30034617fac") : secret "insights-runtime-extractor-tls" not found
Sep 19 10:37:19.561513 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:19.561479    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g5qhx_bd12ed6b-5433-4b19-9eba-1973e0bd4afa/serve-healthcheck-canary/0.log"
Sep 19 10:37:21.201726 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:21.201682    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls\") pod \"dns-default-wf45t\" (UID: \"daad582d-23b8-445c-8c99-dec7c11c95a7\") " pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:21.202161 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:21.201864    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 10:37:21.202161 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:21.201940    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls podName:daad582d-23b8-445c-8c99-dec7c11c95a7 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:37.20192271 +0000 UTC m=+66.698228779 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls") pod "dns-default-wf45t" (UID: "daad582d-23b8-445c-8c99-dec7c11c95a7") : secret "dns-default-metrics-tls" not found
Sep 19 10:37:22.614033 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:22.613987    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:22.614508 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:22.614088    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:22.614508 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:22.614132    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-lntx2\" (UID: \"0b67d17d-e9f0-4b5b-a15f-760f130cae93\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"
Sep 19 10:37:22.614508 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:22.614148    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 19 10:37:22.614508 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:22.614166    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-qzr7f\" (UID: \"3755924c-2290-44b0-ad8f-76c6c14c2b27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f"
Sep 19 10:37:22.614508 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:22.614232    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs podName:18495139-9a71-4bd7-b279-1d7e6470271f nodeName:}" failed. No retries permitted until 2026-09-19 10:37:38.614211878 +0000 UTC m=+68.110517961 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs") pod "router-default-6ddf8bf8dd-66fk8" (UID: "18495139-9a71-4bd7-b279-1d7e6470271f") : secret "router-metrics-certs-default" not found
Sep 19 10:37:22.614508 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:22.614234    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 10:37:22.614508 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:22.614262    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle podName:18495139-9a71-4bd7-b279-1d7e6470271f nodeName:}" failed. No retries permitted until 2026-09-19 10:37:38.614252992 +0000 UTC m=+68.110559062 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle") pod "router-default-6ddf8bf8dd-66fk8" (UID: "18495139-9a71-4bd7-b279-1d7e6470271f") : configmap references non-existent config key: service-ca.crt
Sep 19 10:37:22.614508 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:22.614283    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 19 10:37:22.614508 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:22.614288    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls podName:0b67d17d-e9f0-4b5b-a15f-760f130cae93 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:38.614271405 +0000 UTC m=+68.110577495 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-lntx2" (UID: "0b67d17d-e9f0-4b5b-a15f-760f130cae93") : secret "cluster-monitoring-operator-tls" not found
Sep 19 10:37:22.614508 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:22.614434    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls podName:3755924c-2290-44b0-ad8f-76c6c14c2b27 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:38.614405876 +0000 UTC m=+68.110711951 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-qzr7f" (UID: "3755924c-2290-44b0-ad8f-76c6c14c2b27") : secret "samples-operator-tls" not found
Sep 19 10:37:22.715419 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:22.715380    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:22.715638 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:22.715551    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 10:37:22.715638 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:22.715589    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77fdd969cd-nq42r: secret "image-registry-tls" not found
Sep 19 10:37:22.715762 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:22.715660    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls podName:39728838-de41-40c4-8d66-362ab1882457 nodeName:}" failed. No retries permitted until 2026-09-19 10:37:38.715638453 +0000 UTC m=+68.211944528 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls") pod "image-registry-77fdd969cd-nq42r" (UID: "39728838-de41-40c4-8d66-362ab1882457") : secret "image-registry-tls" not found
Sep 19 10:37:23.522901 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:23.522865    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:23.523208 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:23.522948    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:23.523208 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:23.523032    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 10:37:23.523208 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:23.523037    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 10:37:23.523208 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:23.523102    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-rbac-proxy-cm podName:2de2ff6b-99b1-4a41-ba5e-a30034617fac nodeName:}" failed. No retries permitted until 2026-09-19 10:37:31.5230866 +0000 UTC m=+61.019392669 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lch4t" (UID: "2de2ff6b-99b1-4a41-ba5e-a30034617fac") : configmap "kube-rbac-proxy" not found
Sep 19 10:37:23.523208 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:23.523127    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls podName:2de2ff6b-99b1-4a41-ba5e-a30034617fac nodeName:}" failed. No retries permitted until 2026-09-19 10:37:31.523109221 +0000 UTC m=+61.019415308 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lch4t" (UID: "2de2ff6b-99b1-4a41-ba5e-a30034617fac") : secret "insights-runtime-extractor-tls" not found
Sep 19 10:37:30.997227 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:30.997191    2576 scope.go:117] "RemoveContainer" containerID="2fe6f844c3faed1f56ca5fef61a847451688670bcebde7eed2c3a479cec18db1"
Sep 19 10:37:31.458860 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:31.458828    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 10:37:31.459120 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:31.458888    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-m25hk" event={"ID":"320e460b-f15a-4da5-82eb-f92a0f756180","Type":"ContainerStarted","Data":"16679d7d17eb89521b98a393d9e5e5093053bdfe87a8ca68e5c2255894278260"}
Sep 19 10:37:31.459321 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:31.459286    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-m25hk"
Sep 19 10:37:31.475475 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:31.475424    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-fd4bb46c-m25hk" podStartSLOduration=17.883155737 podStartE2EDuration="25.475408151s" totalImagesPullingTime="7.592252414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:37:31.474079483 +0000 UTC m=+60.970385573" watchObservedRunningTime="2026-09-19 10:37:31.475408151 +0000 UTC m=+60.971714242"
Sep 19 10:37:31.543208 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:31.543179    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-fd4bb46c-m25hk"
Sep 19 10:37:31.603913 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:31.603870    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:31.604330 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:31.604310    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:31.604457 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:31.604438    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 10:37:31.604512 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:31.604493    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2de2ff6b-99b1-4a41-ba5e-a30034617fac-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:31.604550 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:37:31.604515    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls podName:2de2ff6b-99b1-4a41-ba5e-a30034617fac nodeName:}" failed. No retries permitted until 2026-09-19 10:37:47.604494164 +0000 UTC m=+77.100800249 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lch4t" (UID: "2de2ff6b-99b1-4a41-ba5e-a30034617fac") : secret "insights-runtime-extractor-tls" not found
Sep 19 10:37:32.296741 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:32.296715    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m9f9w"
Sep 19 10:37:36.649091 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:36.649030    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs\") pod \"network-metrics-daemon-7r6w2\" (UID: \"64bed023-f7cc-46f8-b5c0-fbb2e62f56f9\") " pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:37:36.651805 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:36.651783    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 19 10:37:36.662331 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:36.662301    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64bed023-f7cc-46f8-b5c0-fbb2e62f56f9-metrics-certs\") pod \"network-metrics-daemon-7r6w2\" (UID: \"64bed023-f7cc-46f8-b5c0-fbb2e62f56f9\") " pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:37:36.749819 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:36.749777    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret\") pod \"global-pull-secret-syncer-75sfv\" (UID: \"acbcb3f6-b65e-4ee4-b497-e5c913e646d7\") " pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:37:36.752328 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:36.752303    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 19 10:37:36.762744 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:36.762717    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/acbcb3f6-b65e-4ee4-b497-e5c913e646d7-original-pull-secret\") pod \"global-pull-secret-syncer-75sfv\" (UID: \"acbcb3f6-b65e-4ee4-b497-e5c913e646d7\") " pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:37:36.850548 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:36.850502    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgsj\" (UniqueName: \"kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj\") pod \"network-check-target-n7g7l\" (UID: \"b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4\") " pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:37:36.853251 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:36.853219    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmgsj\" (UniqueName: \"kubernetes.io/projected/b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4-kube-api-access-wmgsj\") pod \"network-check-target-n7g7l\" (UID: \"b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4\") " pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:37:36.907394 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:36.907295    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-75sfv"
Sep 19 10:37:36.915858 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:36.915828    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ql7r7\""
Sep 19 10:37:36.920796 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:36.920777    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6z24q\""
Sep 19 10:37:36.923632 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:36.923616    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:37:36.929826 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:36.929812    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7r6w2"
Sep 19 10:37:37.212887 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:37.212678    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-75sfv"]
Sep 19 10:37:37.215696 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:37:37.215649    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacbcb3f6_b65e_4ee4_b497_e5c913e646d7.slice/crio-44a60d595f9a6ff4c8f8aaf857d91d62fa2e83929c2ab166f2fcb09bebd26f4c WatchSource:0}: Error finding container 44a60d595f9a6ff4c8f8aaf857d91d62fa2e83929c2ab166f2fcb09bebd26f4c: Status 404 returned error can't find the container with id 44a60d595f9a6ff4c8f8aaf857d91d62fa2e83929c2ab166f2fcb09bebd26f4c
Sep 19 10:37:37.239720 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:37.239687    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n7g7l"]
Sep 19 10:37:37.242282 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:37:37.242251    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7604a22_7baf_4c57_8c3a_5cf1ca86f7a4.slice/crio-0775af3e00e022f5d00d1106705314015eb250eb4ea556259a446efb1517323c WatchSource:0}: Error finding container 0775af3e00e022f5d00d1106705314015eb250eb4ea556259a446efb1517323c: Status 404 returned error can't find the container with id 0775af3e00e022f5d00d1106705314015eb250eb4ea556259a446efb1517323c
Sep 19 10:37:37.254959 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:37.254930    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls\") pod \"dns-default-wf45t\" (UID: \"daad582d-23b8-445c-8c99-dec7c11c95a7\") " pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:37.257578 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:37.257555    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/daad582d-23b8-445c-8c99-dec7c11c95a7-metrics-tls\") pod \"dns-default-wf45t\" (UID: \"daad582d-23b8-445c-8c99-dec7c11c95a7\") " pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:37.271798 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:37.271732    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7r6w2"]
Sep 19 10:37:37.274259 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:37:37.274226    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64bed023_f7cc_46f8_b5c0_fbb2e62f56f9.slice/crio-93c56e30ecb3cb1b60a8b8ef8ec121d41c4501357153262660b4bf71540bbdc4 WatchSource:0}: Error finding container 93c56e30ecb3cb1b60a8b8ef8ec121d41c4501357153262660b4bf71540bbdc4: Status 404 returned error can't find the container with id 93c56e30ecb3cb1b60a8b8ef8ec121d41c4501357153262660b4bf71540bbdc4
Sep 19 10:37:37.418488 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:37.418453    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qlvz2\""
Sep 19 10:37:37.427453 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:37.427423    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:37.481734 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:37.481694    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7r6w2" event={"ID":"64bed023-f7cc-46f8-b5c0-fbb2e62f56f9","Type":"ContainerStarted","Data":"93c56e30ecb3cb1b60a8b8ef8ec121d41c4501357153262660b4bf71540bbdc4"}
Sep 19 10:37:37.483437 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:37.483407    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n7g7l" event={"ID":"b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4","Type":"ContainerStarted","Data":"4305b938dab8869c2918ccbc9320b67aceb9ae9681e0d4e3360235d1294742cd"}
Sep 19 10:37:37.483618 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:37.483460    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n7g7l" event={"ID":"b7604a22-7baf-4c57-8c3a-5cf1ca86f7a4","Type":"ContainerStarted","Data":"0775af3e00e022f5d00d1106705314015eb250eb4ea556259a446efb1517323c"}
Sep 19 10:37:37.483618 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:37.483610    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:37:37.484936 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:37.484912    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-75sfv" event={"ID":"acbcb3f6-b65e-4ee4-b497-e5c913e646d7","Type":"ContainerStarted","Data":"44a60d595f9a6ff4c8f8aaf857d91d62fa2e83929c2ab166f2fcb09bebd26f4c"}
Sep 19 10:37:37.498951 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:37.498890    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-n7g7l" podStartSLOduration=66.498869217 podStartE2EDuration="1m6.498869217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:36:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:37:37.497634984 +0000 UTC m=+66.993941077" watchObservedRunningTime="2026-09-19 10:37:37.498869217 +0000 UTC m=+66.995175311"
Sep 19 10:37:37.735457 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:37.735368    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wf45t"]
Sep 19 10:37:37.738312 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:37:37.738282    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaad582d_23b8_445c_8c99_dec7c11c95a7.slice/crio-dde4d6823cdbd4ec12538ec75676ca2f27dc9f39d2e64a3fa53a5f3adda634a0 WatchSource:0}: Error finding container dde4d6823cdbd4ec12538ec75676ca2f27dc9f39d2e64a3fa53a5f3adda634a0: Status 404 returned error can't find the container with id dde4d6823cdbd4ec12538ec75676ca2f27dc9f39d2e64a3fa53a5f3adda634a0
Sep 19 10:37:38.492173 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:38.492118    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wf45t" event={"ID":"daad582d-23b8-445c-8c99-dec7c11c95a7","Type":"ContainerStarted","Data":"dde4d6823cdbd4ec12538ec75676ca2f27dc9f39d2e64a3fa53a5f3adda634a0"}
Sep 19 10:37:38.669254 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:38.668519    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:38.669254 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:38.668576    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-lntx2\" (UID: \"0b67d17d-e9f0-4b5b-a15f-760f130cae93\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"
Sep 19 10:37:38.669254 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:38.668607    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-qzr7f\" (UID: \"3755924c-2290-44b0-ad8f-76c6c14c2b27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f"
Sep 19 10:37:38.669254 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:38.668681    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:38.670230 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:38.670163    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/18495139-9a71-4bd7-b279-1d7e6470271f-service-ca-bundle\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:38.672629 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:38.672589    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18495139-9a71-4bd7-b279-1d7e6470271f-metrics-certs\") pod \"router-default-6ddf8bf8dd-66fk8\" (UID: \"18495139-9a71-4bd7-b279-1d7e6470271f\") " pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:38.674032 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:38.673989    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0b67d17d-e9f0-4b5b-a15f-760f130cae93-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-lntx2\" (UID: \"0b67d17d-e9f0-4b5b-a15f-760f130cae93\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"
Sep 19 10:37:38.674638 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:38.674599    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3755924c-2290-44b0-ad8f-76c6c14c2b27-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-qzr7f\" (UID: \"3755924c-2290-44b0-ad8f-76c6c14c2b27\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f"
Sep 19 10:37:38.772440 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:38.772226    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:38.775856 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:38.775799    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls\") pod \"image-registry-77fdd969cd-nq42r\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") " pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:38.882255 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:38.882219    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4mppq\""
Sep 19 10:37:38.890479 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:38.890438    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"
Sep 19 10:37:38.892496 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:38.892468    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6gjk5\""
Sep 19 10:37:38.898798 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:38.898773    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-j7rmx\""
Sep 19 10:37:38.900537 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:38.900508    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f"
Sep 19 10:37:38.906559 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:38.906531    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:39.000228 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:39.000143    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lp9gc\""
Sep 19 10:37:39.008389 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:39.008353    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:39.167790 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:39.167750    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2"]
Sep 19 10:37:39.444499 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:37:39.444467    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b67d17d_e9f0_4b5b_a15f_760f130cae93.slice/crio-05bb04e2864b2133f8228658e77ee4befcd0ef8e3cf8a2202ecf58ce982ed4b0 WatchSource:0}: Error finding container 05bb04e2864b2133f8228658e77ee4befcd0ef8e3cf8a2202ecf58ce982ed4b0: Status 404 returned error can't find the container with id 05bb04e2864b2133f8228658e77ee4befcd0ef8e3cf8a2202ecf58ce982ed4b0
Sep 19 10:37:39.498986 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:39.498948    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2" event={"ID":"0b67d17d-e9f0-4b5b-a15f-760f130cae93","Type":"ContainerStarted","Data":"05bb04e2864b2133f8228658e77ee4befcd0ef8e3cf8a2202ecf58ce982ed4b0"}
Sep 19 10:37:39.501259 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:39.501219    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7r6w2" event={"ID":"64bed023-f7cc-46f8-b5c0-fbb2e62f56f9","Type":"ContainerStarted","Data":"d1112913090a163a0fa34886ca71ef2770101e0de70bb33438165ee41c4b9b5a"}
Sep 19 10:37:39.501426 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:39.501287    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7r6w2" event={"ID":"64bed023-f7cc-46f8-b5c0-fbb2e62f56f9","Type":"ContainerStarted","Data":"3b31d5860fdaec6221ed9221615aeeef4f6773cf4326bf78385bedab65c28402"}
Sep 19 10:37:39.528386 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:39.528240    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7r6w2" podStartSLOduration=67.271502324 podStartE2EDuration="1m8.528214452s" totalImagesPullingTime="1.256712128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:37:39.527565548 +0000 UTC m=+69.023871639" watchObservedRunningTime="2026-09-19 10:37:39.528214452 +0000 UTC m=+69.024520545"
Sep 19 10:37:39.732177 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:39.731108    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6ddf8bf8dd-66fk8"]
Sep 19 10:37:39.737035 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:37:39.734875    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18495139_9a71_4bd7_b279_1d7e6470271f.slice/crio-cd63c3391b90e338166df16dc0cf314e166aaca942587005a7719abe51ab6e31 WatchSource:0}: Error finding container cd63c3391b90e338166df16dc0cf314e166aaca942587005a7719abe51ab6e31: Status 404 returned error can't find the container with id cd63c3391b90e338166df16dc0cf314e166aaca942587005a7719abe51ab6e31
Sep 19 10:37:39.767145 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:39.767100    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f"]
Sep 19 10:37:39.773180 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:39.773117    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77fdd969cd-nq42r"]
Sep 19 10:37:39.777032 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:37:39.776996    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39728838_de41_40c4_8d66_362ab1882457.slice/crio-cda4db82d9cdc9b56197750ab1e104a5312b8b14dc0ba8f592b62ade244c2221 WatchSource:0}: Error finding container cda4db82d9cdc9b56197750ab1e104a5312b8b14dc0ba8f592b62ade244c2221: Status 404 returned error can't find the container with id cda4db82d9cdc9b56197750ab1e104a5312b8b14dc0ba8f592b62ade244c2221
Sep 19 10:37:40.511290 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:40.511249    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8" event={"ID":"18495139-9a71-4bd7-b279-1d7e6470271f","Type":"ContainerStarted","Data":"075efb17ace2ceb4b77898a3309b283176bed567fb64e7a8478c20d791647aa0"}
Sep 19 10:37:40.511487 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:40.511355    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8" event={"ID":"18495139-9a71-4bd7-b279-1d7e6470271f","Type":"ContainerStarted","Data":"cd63c3391b90e338166df16dc0cf314e166aaca942587005a7719abe51ab6e31"}
Sep 19 10:37:40.514567 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:40.514532    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f" event={"ID":"3755924c-2290-44b0-ad8f-76c6c14c2b27","Type":"ContainerStarted","Data":"c00611c1123273bb53ceff1c3df8a0a2c9b6d042bfa25d0f1e50778cd50bb679"}
Sep 19 10:37:40.518245 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:40.518211    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wf45t" event={"ID":"daad582d-23b8-445c-8c99-dec7c11c95a7","Type":"ContainerStarted","Data":"be79b060941c848207a0b20d1d24f48b0dc38654c0f9035424e0314f507b9bf2"}
Sep 19 10:37:40.518406 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:40.518276    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wf45t" event={"ID":"daad582d-23b8-445c-8c99-dec7c11c95a7","Type":"ContainerStarted","Data":"e92ad310d2036c5add02ba0529424bd47bc0a088afd45306535d7a2d1af34802"}
Sep 19 10:37:40.518406 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:40.518310    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:40.521485 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:40.521448    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77fdd969cd-nq42r" event={"ID":"39728838-de41-40c4-8d66-362ab1882457","Type":"ContainerStarted","Data":"41b071768b61bb0385383ba82a53139f49b80e20f2dc0b0468f5ebbb669ed946"}
Sep 19 10:37:40.521612 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:40.521512    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77fdd969cd-nq42r" event={"ID":"39728838-de41-40c4-8d66-362ab1882457","Type":"ContainerStarted","Data":"cda4db82d9cdc9b56197750ab1e104a5312b8b14dc0ba8f592b62ade244c2221"}
Sep 19 10:37:40.522085 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:40.522065    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:37:40.548877 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:40.548809    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8" podStartSLOduration=34.54878461 podStartE2EDuration="34.54878461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:37:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:37:40.531689213 +0000 UTC m=+70.027995312" watchObservedRunningTime="2026-09-19 10:37:40.54878461 +0000 UTC m=+70.045090712"
Sep 19 10:37:40.549725 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:40.549456    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wf45t" podStartSLOduration=33.811503991 podStartE2EDuration="35.549438185s" totalImagesPullingTime="1.737934194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:37:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:37:40.547989089 +0000 UTC m=+70.044295188" watchObservedRunningTime="2026-09-19 10:37:40.549438185 +0000 UTC m=+70.045744287"
Sep 19 10:37:40.566016 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:40.565874    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77fdd969cd-nq42r" podStartSLOduration=34.565848218 podStartE2EDuration="34.565848218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:37:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:37:40.564444498 +0000 UTC m=+70.060750588" watchObservedRunningTime="2026-09-19 10:37:40.565848218 +0000 UTC m=+70.062154320"
Sep 19 10:37:40.907679 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:40.907640    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:40.911095 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:40.910860    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:41.527775 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:41.527740    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:41.529113 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:41.529085    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77fdd969cd-nq42r"]
Sep 19 10:37:41.529263 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:41.529198    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6ddf8bf8dd-66fk8"
Sep 19 10:37:43.536691 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:43.536652    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f" event={"ID":"3755924c-2290-44b0-ad8f-76c6c14c2b27","Type":"ContainerStarted","Data":"43fcaf86b6e263b0b3f2cf326d80ff9b08aabb2643fe8de0285e6d49a0caf314"}
Sep 19 10:37:43.537286 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:43.536713    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f" event={"ID":"3755924c-2290-44b0-ad8f-76c6c14c2b27","Type":"ContainerStarted","Data":"fc8e9ab06212b970a61aaa1786b2414202e326025b461802ef446652a19f5dc9"}
Sep 19 10:37:43.538554 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:43.538524    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-75sfv" event={"ID":"acbcb3f6-b65e-4ee4-b497-e5c913e646d7","Type":"ContainerStarted","Data":"a17d293ca72bb0539e95d187b07594911408f4463133ce0fb82bc209adc7f6f2"}
Sep 19 10:37:43.540311 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:43.540279    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2" event={"ID":"0b67d17d-e9f0-4b5b-a15f-760f130cae93","Type":"ContainerStarted","Data":"de07803621989b8e0be90508d88c87acda634c86e678273504635f5ca6f599f2"}
Sep 19 10:37:43.552292 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:43.552234    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-qzr7f" podStartSLOduration=34.467126058 podStartE2EDuration="37.552215572s" totalImagesPullingTime="3.085089514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:37:43.551549639 +0000 UTC m=+73.047855732" watchObservedRunningTime="2026-09-19 10:37:43.552215572 +0000 UTC m=+73.048522000"
Sep 19 10:37:43.570586 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:43.570526    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-lntx2" podStartSLOduration=34.09745374 podStartE2EDuration="37.570507219s" totalImagesPullingTime="3.473053479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:37:43.569788598 +0000 UTC m=+73.066094690" watchObservedRunningTime="2026-09-19 10:37:43.570507219 +0000 UTC m=+73.066813364"
Sep 19 10:37:43.601756 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:43.601696    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-75sfv" podStartSLOduration=66.900254096 podStartE2EDuration="1m12.601670855s" totalImagesPullingTime="5.701416759s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:37:43.600368546 +0000 UTC m=+73.096674637" watchObservedRunningTime="2026-09-19 10:37:43.601670855 +0000 UTC m=+73.097976946"
Sep 19 10:37:47.656934 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:47.656882    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:47.659464 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:47.659441    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2de2ff6b-99b1-4a41-ba5e-a30034617fac-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lch4t\" (UID: \"2de2ff6b-99b1-4a41-ba5e-a30034617fac\") " pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:47.920224 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:47.920136    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5c6t4\""
Sep 19 10:37:47.928211 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:47.928177    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lch4t"
Sep 19 10:37:48.229323 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:48.229246    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lch4t"]
Sep 19 10:37:48.232404 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:37:48.232372    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2de2ff6b_99b1_4a41_ba5e_a30034617fac.slice/crio-146ebbf7e32c03c11a699e6857fbc324477e7f653562204490a16f4acc5d3fbf WatchSource:0}: Error finding container 146ebbf7e32c03c11a699e6857fbc324477e7f653562204490a16f4acc5d3fbf: Status 404 returned error can't find the container with id 146ebbf7e32c03c11a699e6857fbc324477e7f653562204490a16f4acc5d3fbf
Sep 19 10:37:48.559381 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:48.559279    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lch4t" event={"ID":"2de2ff6b-99b1-4a41-ba5e-a30034617fac","Type":"ContainerStarted","Data":"a6494acc55f4e3b9266943f4c3e599d645664e80874a80c680232ebe1af0aba4"}
Sep 19 10:37:48.559381 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:48.559345    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lch4t" event={"ID":"2de2ff6b-99b1-4a41-ba5e-a30034617fac","Type":"ContainerStarted","Data":"146ebbf7e32c03c11a699e6857fbc324477e7f653562204490a16f4acc5d3fbf"}
Sep 19 10:37:49.567359 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:49.567320    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lch4t" event={"ID":"2de2ff6b-99b1-4a41-ba5e-a30034617fac","Type":"ContainerStarted","Data":"13ef2776703846ea03ac93a309fc8269a315d284f38595eec7a4a0cec19c58eb"}
Sep 19 10:37:50.436035 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.436002    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7dc6f799bf-9zsks"]
Sep 19 10:37:50.438241 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.438218    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.441623 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.441596    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 19 10:37:50.441903 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.441885    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 19 10:37:50.442057 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.441887    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 19 10:37:50.443493 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.443023    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 19 10:37:50.443493 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.443144    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 19 10:37:50.443493 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.443267    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 19 10:37:50.443493 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.443404    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-f4tr5\""
Sep 19 10:37:50.443493 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.443488    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 19 10:37:50.447865 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.447785    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 19 10:37:50.468503 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.468463    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dc6f799bf-9zsks"]
Sep 19 10:37:50.529839 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.529806    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wf45t"
Sep 19 10:37:50.583322 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.583283    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-trusted-ca-bundle\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.583830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.583366    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-console-config\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.583830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.583428    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c344464-353e-4403-aa89-2522f7a4a886-console-serving-cert\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.583830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.583453    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-service-ca\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.583830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.583492    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c344464-353e-4403-aa89-2522f7a4a886-console-oauth-config\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.583830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.583527    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt42r\" (UniqueName: \"kubernetes.io/projected/4c344464-353e-4403-aa89-2522f7a4a886-kube-api-access-gt42r\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.583830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.583556    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-oauth-serving-cert\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.684988 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.684947    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c344464-353e-4403-aa89-2522f7a4a886-console-oauth-config\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.684988 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.684990    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt42r\" (UniqueName: \"kubernetes.io/projected/4c344464-353e-4403-aa89-2522f7a4a886-kube-api-access-gt42r\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.685253 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.685020    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-oauth-serving-cert\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.685253 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.685084    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-trusted-ca-bundle\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.685253 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.685133    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-console-config\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.685253 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.685199    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c344464-353e-4403-aa89-2522f7a4a886-console-serving-cert\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.685253 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.685226    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-service-ca\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.686117 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.686020    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-oauth-serving-cert\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.686589 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.686570    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-console-config\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.686659 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.686622    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-trusted-ca-bundle\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.687168 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.687151    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-service-ca\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.688472 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.688437    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c344464-353e-4403-aa89-2522f7a4a886-console-oauth-config\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.688609 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.688532    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c344464-353e-4403-aa89-2522f7a4a886-console-serving-cert\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.698150 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.698119    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt42r\" (UniqueName: \"kubernetes.io/projected/4c344464-353e-4403-aa89-2522f7a4a886-kube-api-access-gt42r\") pod \"console-7dc6f799bf-9zsks\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") " pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:50.752500 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:50.752468    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:37:51.034094 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:51.033985    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dc6f799bf-9zsks"]
Sep 19 10:37:51.036836 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:37:51.036811    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c344464_353e_4403_aa89_2522f7a4a886.slice/crio-8f776a1d4377fa90d800ee56e02a3fa313d022a462ea7fab4e3b12151eb4cb3b WatchSource:0}: Error finding container 8f776a1d4377fa90d800ee56e02a3fa313d022a462ea7fab4e3b12151eb4cb3b: Status 404 returned error can't find the container with id 8f776a1d4377fa90d800ee56e02a3fa313d022a462ea7fab4e3b12151eb4cb3b
Sep 19 10:37:51.581002 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:51.580957    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lch4t" event={"ID":"2de2ff6b-99b1-4a41-ba5e-a30034617fac","Type":"ContainerStarted","Data":"0e72d8930a30d2431293fcc7ea1578c1f45bf4469e0258d33112caf0433e448d"}
Sep 19 10:37:51.583829 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:51.583764    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc6f799bf-9zsks" event={"ID":"4c344464-353e-4403-aa89-2522f7a4a886","Type":"ContainerStarted","Data":"8f776a1d4377fa90d800ee56e02a3fa313d022a462ea7fab4e3b12151eb4cb3b"}
Sep 19 10:37:51.600833 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:51.600763    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lch4t" podStartSLOduration=34.326750081 podStartE2EDuration="36.600738449s" totalImagesPullingTime="2.273988368s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:37:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:37:51.599240159 +0000 UTC m=+81.095546261" watchObservedRunningTime="2026-09-19 10:37:51.600738449 +0000 UTC m=+81.097044552"
Sep 19 10:37:55.602944 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:55.602889    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc6f799bf-9zsks" event={"ID":"4c344464-353e-4403-aa89-2522f7a4a886","Type":"ContainerStarted","Data":"4981446a9bba8edc2c8a5f23550b287fb1b28929b1a11e9cef5b6302950bc6e7"}
Sep 19 10:37:55.621931 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:37:55.621874    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7dc6f799bf-9zsks" podStartSLOduration=1.7942268160000001 podStartE2EDuration="5.621857663s" totalImagesPullingTime="3.827630847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:37:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:37:55.62103497 +0000 UTC m=+85.117341064" watchObservedRunningTime="2026-09-19 10:37:55.621857663 +0000 UTC m=+85.118163755"
Sep 19 10:38:00.753773 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:00.753726    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:38:00.754213 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:00.753802    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:38:00.758843 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:00.758815    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:38:01.628619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:01.628588    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:38:03.546472 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:03.546442    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:38:05.903991 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:05.903947    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wz4ff"]
Sep 19 10:38:05.907070 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:05.907026    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:05.911749 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:05.911721    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 19 10:38:05.911904 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:05.911843    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 19 10:38:05.911904 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:05.911846    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 19 10:38:05.912002 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:05.911898    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 19 10:38:05.912002 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:05.911900    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-c42jv\""
Sep 19 10:38:06.004889 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.004850    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d38fa806-49b5-4869-87f5-2eeca2e1db69-node-exporter-accelerators-collector-config\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.004889 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.004898    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d38fa806-49b5-4869-87f5-2eeca2e1db69-root\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.005176 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.004978    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d38fa806-49b5-4869-87f5-2eeca2e1db69-node-exporter-wtmp\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.005176 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.005067    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d38fa806-49b5-4869-87f5-2eeca2e1db69-metrics-client-ca\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.005176 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.005097    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkwg2\" (UniqueName: \"kubernetes.io/projected/d38fa806-49b5-4869-87f5-2eeca2e1db69-kube-api-access-qkwg2\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.005176 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.005127    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d38fa806-49b5-4869-87f5-2eeca2e1db69-sys\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.005176 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.005148    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d38fa806-49b5-4869-87f5-2eeca2e1db69-node-exporter-textfile\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.005339 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.005218    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d38fa806-49b5-4869-87f5-2eeca2e1db69-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.005339 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.005272    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d38fa806-49b5-4869-87f5-2eeca2e1db69-node-exporter-tls\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.106448 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.106413    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d38fa806-49b5-4869-87f5-2eeca2e1db69-node-exporter-accelerators-collector-config\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.106448 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.106453    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d38fa806-49b5-4869-87f5-2eeca2e1db69-root\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.106690 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.106478    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d38fa806-49b5-4869-87f5-2eeca2e1db69-node-exporter-wtmp\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.106690 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.106504    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d38fa806-49b5-4869-87f5-2eeca2e1db69-metrics-client-ca\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.106690 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.106559    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d38fa806-49b5-4869-87f5-2eeca2e1db69-root\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.106690 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.106621    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkwg2\" (UniqueName: \"kubernetes.io/projected/d38fa806-49b5-4869-87f5-2eeca2e1db69-kube-api-access-qkwg2\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.106690 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.106674    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d38fa806-49b5-4869-87f5-2eeca2e1db69-sys\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.106925 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.106627    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d38fa806-49b5-4869-87f5-2eeca2e1db69-node-exporter-wtmp\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.106925 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.106706    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d38fa806-49b5-4869-87f5-2eeca2e1db69-node-exporter-textfile\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.106925 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.106758    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d38fa806-49b5-4869-87f5-2eeca2e1db69-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.106925 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.106766    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d38fa806-49b5-4869-87f5-2eeca2e1db69-sys\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.106925 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.106794    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d38fa806-49b5-4869-87f5-2eeca2e1db69-node-exporter-tls\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.106925 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:38:06.106908    2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 19 10:38:06.107246 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.106942    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d38fa806-49b5-4869-87f5-2eeca2e1db69-node-exporter-textfile\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.107246 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:38:06.106977    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d38fa806-49b5-4869-87f5-2eeca2e1db69-node-exporter-tls podName:d38fa806-49b5-4869-87f5-2eeca2e1db69 nodeName:}" failed. No retries permitted until 2026-09-19 10:38:06.606956017 +0000 UTC m=+96.103262114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d38fa806-49b5-4869-87f5-2eeca2e1db69-node-exporter-tls") pod "node-exporter-wz4ff" (UID: "d38fa806-49b5-4869-87f5-2eeca2e1db69") : secret "node-exporter-tls" not found
Sep 19 10:38:06.107246 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.107161    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d38fa806-49b5-4869-87f5-2eeca2e1db69-metrics-client-ca\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.107246 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.107185    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d38fa806-49b5-4869-87f5-2eeca2e1db69-node-exporter-accelerators-collector-config\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.109298 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.109276    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d38fa806-49b5-4869-87f5-2eeca2e1db69-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.117549 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.117512    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkwg2\" (UniqueName: \"kubernetes.io/projected/d38fa806-49b5-4869-87f5-2eeca2e1db69-kube-api-access-qkwg2\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.611062 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.611003    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d38fa806-49b5-4869-87f5-2eeca2e1db69-node-exporter-tls\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.613667 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.613637    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d38fa806-49b5-4869-87f5-2eeca2e1db69-node-exporter-tls\") pod \"node-exporter-wz4ff\" (UID: \"d38fa806-49b5-4869-87f5-2eeca2e1db69\") " pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.816911 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.816867    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wz4ff"
Sep 19 10:38:06.827731 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:38:06.827694    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd38fa806_49b5_4869_87f5_2eeca2e1db69.slice/crio-77253edfe0960e4f1cf9f3fbada58dcf7a4905aa8f1457e02a574d4c7cfd0000 WatchSource:0}: Error finding container 77253edfe0960e4f1cf9f3fbada58dcf7a4905aa8f1457e02a574d4c7cfd0000: Status 404 returned error can't find the container with id 77253edfe0960e4f1cf9f3fbada58dcf7a4905aa8f1457e02a574d4c7cfd0000
Sep 19 10:38:06.983603 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.983518    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 19 10:38:06.986630 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.986606    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:06.988855 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.988832    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 19 10:38:06.989007 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.988968    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xzc8p\""
Sep 19 10:38:06.989182 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.989162    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 19 10:38:06.989389 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.989363    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 19 10:38:06.989389 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.989390    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 19 10:38:06.989565 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.989408    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 19 10:38:06.989565 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.989410    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 19 10:38:06.989565 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.989520    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 19 10:38:06.989565 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.989545    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 19 10:38:06.990196 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:06.990173    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 19 10:38:07.001994 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.001961    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 19 10:38:07.014655 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.014615    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.014655 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.014660    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.014920 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.014724    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-web-config\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.014920 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.014758    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.014920 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.014784    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.014920 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.014810    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.014920 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.014829    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6nbg\" (UniqueName: \"kubernetes.io/projected/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-kube-api-access-v6nbg\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.014920 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.014861    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.014920 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.014878    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.014920 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.014906    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-config-volume\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.015383 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.014946    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.015383 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.014982    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-config-out\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.015383 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.015014    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.115819 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.115776    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.115819 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.115825    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.116121 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.115850    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-web-config\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.116121 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.115872    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.116121 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.115898    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.116121 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.115928    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.116121 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.115950    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6nbg\" (UniqueName: \"kubernetes.io/projected/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-kube-api-access-v6nbg\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.116121 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.115985    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.116121 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.116007    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.116441 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.116252    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-config-volume\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.116441 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.116311    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.116441 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.116364    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-config-out\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.116441 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.116425    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.117247 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.117209    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.118320 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.118273    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.118611 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.118588    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.120196 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.119709    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-web-config\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.120196 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.119934    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-config-volume\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.120196 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.120159    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.120464 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.120441    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.120538 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.120438    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.121095 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.121071    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.121738 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.121714    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.121843 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.121767    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-config-out\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.121843 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.121823    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.123749 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.123730    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6nbg\" (UniqueName: \"kubernetes.io/projected/ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f-kube-api-access-v6nbg\") pod \"alertmanager-main-0\" (UID: \"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.298408 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.298300    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 19 10:38:07.583104 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.582986    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 19 10:38:07.585514 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:38:07.585475    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebfd45d9_bf0b_40fd_8d7c_3e1c5978722f.slice/crio-b04ab6f69b23a274abd6f557b597084ceb81f02da849fe18b5a951674e3a7aac WatchSource:0}: Error finding container b04ab6f69b23a274abd6f557b597084ceb81f02da849fe18b5a951674e3a7aac: Status 404 returned error can't find the container with id b04ab6f69b23a274abd6f557b597084ceb81f02da849fe18b5a951674e3a7aac
Sep 19 10:38:07.648213 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.648167    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f","Type":"ContainerStarted","Data":"b04ab6f69b23a274abd6f557b597084ceb81f02da849fe18b5a951674e3a7aac"}
Sep 19 10:38:07.649689 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:07.649654    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wz4ff" event={"ID":"d38fa806-49b5-4869-87f5-2eeca2e1db69","Type":"ContainerStarted","Data":"77253edfe0960e4f1cf9f3fbada58dcf7a4905aa8f1457e02a574d4c7cfd0000"}
Sep 19 10:38:08.496222 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.496183    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-n7g7l"
Sep 19 10:38:08.561406 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.561100    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-77fdd969cd-nq42r" podUID="39728838-de41-40c4-8d66-362ab1882457" containerName="registry" containerID="cri-o://41b071768b61bb0385383ba82a53139f49b80e20f2dc0b0468f5ebbb669ed946" gracePeriod=30
Sep 19 10:38:08.655337 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.655299    2576 generic.go:356] "Generic (PLEG): container finished" podID="d38fa806-49b5-4869-87f5-2eeca2e1db69" containerID="8f39676677faa7110526e0c77812a65dae3cd38fb6939f7023852ac94d15256a" exitCode=0
Sep 19 10:38:08.655764 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.655372    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wz4ff" event={"ID":"d38fa806-49b5-4869-87f5-2eeca2e1db69","Type":"ContainerDied","Data":"8f39676677faa7110526e0c77812a65dae3cd38fb6939f7023852ac94d15256a"}
Sep 19 10:38:08.816189 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.816163    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:38:08.895591 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.895557    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-79789845bb-wzxrh"]
Sep 19 10:38:08.895877 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.895865    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39728838-de41-40c4-8d66-362ab1882457" containerName="registry"
Sep 19 10:38:08.895935 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.895879    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="39728838-de41-40c4-8d66-362ab1882457" containerName="registry"
Sep 19 10:38:08.896017 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.895941    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="39728838-de41-40c4-8d66-362ab1882457" containerName="registry"
Sep 19 10:38:08.899784 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.899758    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:08.902287 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.902238    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 19 10:38:08.902287 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.902252    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-195je0f095p4k\""
Sep 19 10:38:08.902539 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.902310    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 19 10:38:08.902539 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.902314    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ls5qv\""
Sep 19 10:38:08.902691 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.902596    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 19 10:38:08.902691 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.902610    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 19 10:38:08.902691 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.902626    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 19 10:38:08.910833 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.910793    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-79789845bb-wzxrh"]
Sep 19 10:38:08.934410 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.934374    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39728838-de41-40c4-8d66-362ab1882457-image-registry-private-configuration\") pod \"39728838-de41-40c4-8d66-362ab1882457\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") "
Sep 19 10:38:08.934604 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.934429    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39728838-de41-40c4-8d66-362ab1882457-registry-certificates\") pod \"39728838-de41-40c4-8d66-362ab1882457\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") "
Sep 19 10:38:08.934604 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.934468    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-bound-sa-token\") pod \"39728838-de41-40c4-8d66-362ab1882457\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") "
Sep 19 10:38:08.934604 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.934512    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7dg2\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-kube-api-access-g7dg2\") pod \"39728838-de41-40c4-8d66-362ab1882457\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") "
Sep 19 10:38:08.934604 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.934535    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls\") pod \"39728838-de41-40c4-8d66-362ab1882457\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") "
Sep 19 10:38:08.934604 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.934574    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39728838-de41-40c4-8d66-362ab1882457-ca-trust-extracted\") pod \"39728838-de41-40c4-8d66-362ab1882457\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") "
Sep 19 10:38:08.934604 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.934602    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39728838-de41-40c4-8d66-362ab1882457-trusted-ca\") pod \"39728838-de41-40c4-8d66-362ab1882457\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") "
Sep 19 10:38:08.934914 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.934627    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39728838-de41-40c4-8d66-362ab1882457-installation-pull-secrets\") pod \"39728838-de41-40c4-8d66-362ab1882457\" (UID: \"39728838-de41-40c4-8d66-362ab1882457\") "
Sep 19 10:38:08.934914 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.934746    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:08.934914 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.934778    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-secret-grpc-tls\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:08.934914 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.934831    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:08.934914 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.934885    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-metrics-client-ca\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:08.935173 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.934929    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:08.935173 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.934956    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4w5r2\" (UniqueName: \"kubernetes.io/projected/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-kube-api-access-4w5r2\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:08.935173 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.934999    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:08.935173 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.935115    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-secret-thanos-querier-tls\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:08.935546 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.934997    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39728838-de41-40c4-8d66-362ab1882457-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "39728838-de41-40c4-8d66-362ab1882457" (UID: "39728838-de41-40c4-8d66-362ab1882457"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:38:08.935602 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.935539    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39728838-de41-40c4-8d66-362ab1882457-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "39728838-de41-40c4-8d66-362ab1882457" (UID: "39728838-de41-40c4-8d66-362ab1882457"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:38:08.937277 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.937244    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39728838-de41-40c4-8d66-362ab1882457-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "39728838-de41-40c4-8d66-362ab1882457" (UID: "39728838-de41-40c4-8d66-362ab1882457"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:38:08.937277 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.937257    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-kube-api-access-g7dg2" (OuterVolumeSpecName: "kube-api-access-g7dg2") pod "39728838-de41-40c4-8d66-362ab1882457" (UID: "39728838-de41-40c4-8d66-362ab1882457"). InnerVolumeSpecName "kube-api-access-g7dg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:38:08.937484 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.937452    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "39728838-de41-40c4-8d66-362ab1882457" (UID: "39728838-de41-40c4-8d66-362ab1882457"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:38:08.937484 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.937454    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39728838-de41-40c4-8d66-362ab1882457-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "39728838-de41-40c4-8d66-362ab1882457" (UID: "39728838-de41-40c4-8d66-362ab1882457"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:38:08.937814 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.937787    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "39728838-de41-40c4-8d66-362ab1882457" (UID: "39728838-de41-40c4-8d66-362ab1882457"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:38:08.943554 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:08.943525    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39728838-de41-40c4-8d66-362ab1882457-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "39728838-de41-40c4-8d66-362ab1882457" (UID: "39728838-de41-40c4-8d66-362ab1882457"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:38:09.035930 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.035892    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-metrics-client-ca\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:09.035930 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.035939    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:09.036240 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.035958    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w5r2\" (UniqueName: \"kubernetes.io/projected/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-kube-api-access-4w5r2\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:09.036240 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.035987    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:09.036240 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.036028    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-secret-thanos-querier-tls\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:09.036240 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.036062    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:09.036240 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.036084    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-secret-grpc-tls\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:09.036240 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.036129    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:09.036240 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.036169    2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-bound-sa-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:38:09.036240 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.036183    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7dg2\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-kube-api-access-g7dg2\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:38:09.036240 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.036198    2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39728838-de41-40c4-8d66-362ab1882457-registry-tls\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:38:09.036240 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.036209    2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39728838-de41-40c4-8d66-362ab1882457-ca-trust-extracted\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:38:09.036240 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.036217    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39728838-de41-40c4-8d66-362ab1882457-trusted-ca\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:38:09.036240 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.036225    2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39728838-de41-40c4-8d66-362ab1882457-installation-pull-secrets\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:38:09.036240 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.036234    2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39728838-de41-40c4-8d66-362ab1882457-image-registry-private-configuration\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:38:09.036240 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.036244    2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39728838-de41-40c4-8d66-362ab1882457-registry-certificates\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:38:09.037127 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.037034    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-metrics-client-ca\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:09.038704 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.038665    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:09.038913 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.038891    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-secret-thanos-querier-tls\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:09.039190 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.039168    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:09.039285 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.039263    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:09.039358 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.039335    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:09.039415 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.039398    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-secret-grpc-tls\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:09.043566 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.043543    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4w5r2\" (UniqueName: \"kubernetes.io/projected/5d46d9af-8b3d-4658-bef1-667d76a7d3f2-kube-api-access-4w5r2\") pod \"thanos-querier-79789845bb-wzxrh\" (UID: \"5d46d9af-8b3d-4658-bef1-667d76a7d3f2\") " pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:09.210730 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.210681    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:09.491424 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.491311    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-79789845bb-wzxrh"]
Sep 19 10:38:09.493463 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:38:09.493427    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d46d9af_8b3d_4658_bef1_667d76a7d3f2.slice/crio-bc13d2f881d24340da5e03a8f1750214cbf44dc69308cbe3f5d2b595c06666b7 WatchSource:0}: Error finding container bc13d2f881d24340da5e03a8f1750214cbf44dc69308cbe3f5d2b595c06666b7: Status 404 returned error can't find the container with id bc13d2f881d24340da5e03a8f1750214cbf44dc69308cbe3f5d2b595c06666b7
Sep 19 10:38:09.661686 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.661653    2576 generic.go:356] "Generic (PLEG): container finished" podID="ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f" containerID="17d845cef5d1e09b39851d44c4c741793272cf3e59420a2c566131be67a75e8f" exitCode=0
Sep 19 10:38:09.662186 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.661732    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f","Type":"ContainerDied","Data":"17d845cef5d1e09b39851d44c4c741793272cf3e59420a2c566131be67a75e8f"}
Sep 19 10:38:09.663175 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.663066    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh" event={"ID":"5d46d9af-8b3d-4658-bef1-667d76a7d3f2","Type":"ContainerStarted","Data":"bc13d2f881d24340da5e03a8f1750214cbf44dc69308cbe3f5d2b595c06666b7"}
Sep 19 10:38:09.664527 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.664499    2576 generic.go:356] "Generic (PLEG): container finished" podID="39728838-de41-40c4-8d66-362ab1882457" containerID="41b071768b61bb0385383ba82a53139f49b80e20f2dc0b0468f5ebbb669ed946" exitCode=0
Sep 19 10:38:09.664656 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.664558    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77fdd969cd-nq42r"
Sep 19 10:38:09.664656 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.664582    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77fdd969cd-nq42r" event={"ID":"39728838-de41-40c4-8d66-362ab1882457","Type":"ContainerDied","Data":"41b071768b61bb0385383ba82a53139f49b80e20f2dc0b0468f5ebbb669ed946"}
Sep 19 10:38:09.664656 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.664648    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77fdd969cd-nq42r" event={"ID":"39728838-de41-40c4-8d66-362ab1882457","Type":"ContainerDied","Data":"cda4db82d9cdc9b56197750ab1e104a5312b8b14dc0ba8f592b62ade244c2221"}
Sep 19 10:38:09.664840 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.664678    2576 scope.go:117] "RemoveContainer" containerID="41b071768b61bb0385383ba82a53139f49b80e20f2dc0b0468f5ebbb669ed946"
Sep 19 10:38:09.667745 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.667717    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wz4ff" event={"ID":"d38fa806-49b5-4869-87f5-2eeca2e1db69","Type":"ContainerStarted","Data":"90c01155d939537d971889e3ca0ecebd85daedc854d0d078e2d84b76a55c4ceb"}
Sep 19 10:38:09.667852 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.667769    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wz4ff" event={"ID":"d38fa806-49b5-4869-87f5-2eeca2e1db69","Type":"ContainerStarted","Data":"063317b6421b327a0c8c8a3e6b01e4e26b2087ad398e663c0c0b41d700a78ee1"}
Sep 19 10:38:09.673856 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.673838    2576 scope.go:117] "RemoveContainer" containerID="41b071768b61bb0385383ba82a53139f49b80e20f2dc0b0468f5ebbb669ed946"
Sep 19 10:38:09.674251 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:38:09.674227    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41b071768b61bb0385383ba82a53139f49b80e20f2dc0b0468f5ebbb669ed946\": container with ID starting with 41b071768b61bb0385383ba82a53139f49b80e20f2dc0b0468f5ebbb669ed946 not found: ID does not exist" containerID="41b071768b61bb0385383ba82a53139f49b80e20f2dc0b0468f5ebbb669ed946"
Sep 19 10:38:09.674332 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.674261    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41b071768b61bb0385383ba82a53139f49b80e20f2dc0b0468f5ebbb669ed946"} err="failed to get container status \"41b071768b61bb0385383ba82a53139f49b80e20f2dc0b0468f5ebbb669ed946\": rpc error: code = NotFound desc = could not find container \"41b071768b61bb0385383ba82a53139f49b80e20f2dc0b0468f5ebbb669ed946\": container with ID starting with 41b071768b61bb0385383ba82a53139f49b80e20f2dc0b0468f5ebbb669ed946 not found: ID does not exist"
Sep 19 10:38:09.699931 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.699892    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77fdd969cd-nq42r"]
Sep 19 10:38:09.703277 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.703247    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-77fdd969cd-nq42r"]
Sep 19 10:38:09.718602 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:09.718541    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wz4ff" podStartSLOduration=3.636539234 podStartE2EDuration="4.718526581s" totalImagesPullingTime="1.01035629s" totalInitContainerRuntime="71.631057ms" isStatefulPod=true podCreationTimestamp="2026-09-19 10:38:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:38:09.717247986 +0000 UTC m=+99.213554080" watchObservedRunningTime="2026-09-19 10:38:09.718526581 +0000 UTC m=+99.214832713"
Sep 19 10:38:10.665399 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.665356    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-pk78w"]
Sep 19 10:38:10.685102 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.685070    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-pk78w"]
Sep 19 10:38:10.685353 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.685135    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-pk78w"
Sep 19 10:38:10.687947 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.687918    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 19 10:38:10.688113 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.688019    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-w4ntb\""
Sep 19 10:38:10.752170 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.752126    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4r67\" (UniqueName: \"kubernetes.io/projected/3bb51bef-73f4-4b3a-9c10-b2eef08ab694-kube-api-access-g4r67\") pod \"monitoring-plugin-5984459ff8-pk78w\" (UID: \"3bb51bef-73f4-4b3a-9c10-b2eef08ab694\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-pk78w"
Sep 19 10:38:10.752556 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.752513    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3bb51bef-73f4-4b3a-9c10-b2eef08ab694-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-pk78w\" (UID: \"3bb51bef-73f4-4b3a-9c10-b2eef08ab694\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-pk78w"
Sep 19 10:38:10.853253 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.853217    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3bb51bef-73f4-4b3a-9c10-b2eef08ab694-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-pk78w\" (UID: \"3bb51bef-73f4-4b3a-9c10-b2eef08ab694\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-pk78w"
Sep 19 10:38:10.853447 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.853285    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4r67\" (UniqueName: \"kubernetes.io/projected/3bb51bef-73f4-4b3a-9c10-b2eef08ab694-kube-api-access-g4r67\") pod \"monitoring-plugin-5984459ff8-pk78w\" (UID: \"3bb51bef-73f4-4b3a-9c10-b2eef08ab694\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-pk78w"
Sep 19 10:38:10.853893 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:38:10.853802    2576 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 19 10:38:10.854323 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:38:10.854302    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bb51bef-73f4-4b3a-9c10-b2eef08ab694-monitoring-plugin-cert podName:3bb51bef-73f4-4b3a-9c10-b2eef08ab694 nodeName:}" failed. No retries permitted until 2026-09-19 10:38:11.353850844 +0000 UTC m=+100.850156915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/3bb51bef-73f4-4b3a-9c10-b2eef08ab694-monitoring-plugin-cert") pod "monitoring-plugin-5984459ff8-pk78w" (UID: "3bb51bef-73f4-4b3a-9c10-b2eef08ab694") : secret "monitoring-plugin-cert" not found
Sep 19 10:38:10.865151 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.865110    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4r67\" (UniqueName: \"kubernetes.io/projected/3bb51bef-73f4-4b3a-9c10-b2eef08ab694-kube-api-access-g4r67\") pod \"monitoring-plugin-5984459ff8-pk78w\" (UID: \"3bb51bef-73f4-4b3a-9c10-b2eef08ab694\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-pk78w"
Sep 19 10:38:10.974557 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.974446    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"]
Sep 19 10:38:10.978159 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.978136    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:10.980942 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.980766    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 19 10:38:10.980942 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.980857    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 19 10:38:10.981201 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.981113    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-b6l5c\""
Sep 19 10:38:10.981201 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.981161    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 19 10:38:10.981308 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.981195    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 19 10:38:10.981478 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.981460    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 19 10:38:10.985784 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.985762    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 19 10:38:10.993011 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:10.992986    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"]
Sep 19 10:38:11.001757 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.001721    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39728838-de41-40c4-8d66-362ab1882457" path="/var/lib/kubelet/pods/39728838-de41-40c4-8d66-362ab1882457/volumes"
Sep 19 10:38:11.054458 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.054424    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-metrics-client-ca\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.054458 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.054465    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-secret-telemeter-client\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.054672 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.054483    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-telemeter-client-tls\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.054672 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.054535    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2fsp\" (UniqueName: \"kubernetes.io/projected/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-kube-api-access-p2fsp\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.054672 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.054626    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.054672 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.054647    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-federate-client-tls\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.054873 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.054677    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-serving-certs-ca-bundle\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.054873 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.054752    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-telemeter-trusted-ca-bundle\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.155309 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.155258    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-telemeter-trusted-ca-bundle\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.155518 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.155315    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-metrics-client-ca\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.155518 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.155350    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-secret-telemeter-client\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.155518 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.155376    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-telemeter-client-tls\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.155518 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.155407    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2fsp\" (UniqueName: \"kubernetes.io/projected/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-kube-api-access-p2fsp\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.155518 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.155474    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.155518 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.155503    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-federate-client-tls\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.156392 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.155545    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-serving-certs-ca-bundle\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.156392 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.156177    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-metrics-client-ca\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.156392 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.156314    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-telemeter-trusted-ca-bundle\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.156392 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.156342    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-serving-certs-ca-bundle\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.158969 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.158919    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-telemeter-client-tls\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.158969 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.158949    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-secret-telemeter-client\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.159149 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.159127    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.159913 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.159885    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-federate-client-tls\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.165141 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.165110    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2fsp\" (UniqueName: \"kubernetes.io/projected/a5d7b77b-8c41-4139-b39f-a277f9a4bc54-kube-api-access-p2fsp\") pod \"telemeter-client-f6f8d447c-5xpgf\" (UID: \"a5d7b77b-8c41-4139-b39f-a277f9a4bc54\") " pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.292520 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.292423    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"
Sep 19 10:38:11.357460 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.357412    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3bb51bef-73f4-4b3a-9c10-b2eef08ab694-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-pk78w\" (UID: \"3bb51bef-73f4-4b3a-9c10-b2eef08ab694\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-pk78w"
Sep 19 10:38:11.360231 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.360205    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3bb51bef-73f4-4b3a-9c10-b2eef08ab694-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-pk78w\" (UID: \"3bb51bef-73f4-4b3a-9c10-b2eef08ab694\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-pk78w"
Sep 19 10:38:11.601763 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:11.601721    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-pk78w"
Sep 19 10:38:12.036521 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.036491    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 19 10:38:12.078037 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.077999    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 19 10:38:12.078237 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.078150    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.080904 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.080867    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8l7967olkbggm\""
Sep 19 10:38:12.081014 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.080938    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 19 10:38:12.081014 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.080948    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 19 10:38:12.081312 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.081285    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 19 10:38:12.081415 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.081285    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 19 10:38:12.081415 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.081347    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 19 10:38:12.081415 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.081371    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 19 10:38:12.081415 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.081384    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 19 10:38:12.081627 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.081474    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 19 10:38:12.081627 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.081534    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 19 10:38:12.081627 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.081578    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 19 10:38:12.082016 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.081999    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 19 10:38:12.082143 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.082024    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mmpmw\""
Sep 19 10:38:12.082143 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.082088    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 19 10:38:12.082143 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.082095    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 19 10:38:12.082143 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.082025    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 19 10:38:12.084727 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.084709    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 19 10:38:12.165820 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.165671    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.165820 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.165719    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.165820 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.165757    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.165820 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.165808    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.166072 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.165848    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-config\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.166072 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.165872    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.166072 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.165898    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b2a07f45-a695-43dc-aba0-35579c4ca70b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.166072 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.165937    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.166392 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.166291    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.166513 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.166413    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-web-config\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.166588 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.166509    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.166588 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.166555    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b2a07f45-a695-43dc-aba0-35579c4ca70b-config-out\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.166740 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.166720    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.166816 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.166757    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.166816 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.166787    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knkcw\" (UniqueName: \"kubernetes.io/projected/b2a07f45-a695-43dc-aba0-35579c4ca70b-kube-api-access-knkcw\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.167034 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.166837    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.167034 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.166859    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b2a07f45-a695-43dc-aba0-35579c4ca70b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.167034 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.166902    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.167034 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.166931    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.167034 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.166952    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.267574 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267537    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.267574 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267580    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.267842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267611    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.267842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267632    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-config\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.267842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267650    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.267842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267667    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b2a07f45-a695-43dc-aba0-35579c4ca70b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.267842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267702    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.267842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267720    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.267842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267738    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-web-config\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.267842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267756    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.267842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267773    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b2a07f45-a695-43dc-aba0-35579c4ca70b-config-out\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.267842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267787    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.267842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267808    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.267842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267826    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knkcw\" (UniqueName: \"kubernetes.io/projected/b2a07f45-a695-43dc-aba0-35579c4ca70b-kube-api-access-knkcw\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.268603 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267855    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.268603 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267871    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b2a07f45-a695-43dc-aba0-35579c4ca70b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.268603 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267890    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.268603 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267904    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.268603 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267918    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.268603 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.267940    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.268603 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.268491    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.270068 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.269402    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.270068 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.269926    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.270941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.270574    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.274072 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.273721    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b2a07f45-a695-43dc-aba0-35579c4ca70b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.274072 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.273998    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.275701 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.275663    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.277169 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.276572    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b2a07f45-a695-43dc-aba0-35579c4ca70b-config-out\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.277169 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.276637    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-web-config\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.277169 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.276772    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.277169 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.276840    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b2a07f45-a695-43dc-aba0-35579c4ca70b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.277169 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.277168    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.277502 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.277476    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.277906 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.277862    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.278193 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.278166    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b2a07f45-a695-43dc-aba0-35579c4ca70b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.279509 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.278553    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-config\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.279509 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.279447    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.279659 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.279608    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.280681 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.280658    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b2a07f45-a695-43dc-aba0-35579c4ca70b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.286428 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.286032    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knkcw\" (UniqueName: \"kubernetes.io/projected/b2a07f45-a695-43dc-aba0-35579c4ca70b-kube-api-access-knkcw\") pod \"prometheus-k8s-0\" (UID: \"b2a07f45-a695-43dc-aba0-35579c4ca70b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.361740 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.361693    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-pk78w"]
Sep 19 10:38:12.366640 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:38:12.366607    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bb51bef_73f4_4b3a_9c10_b2eef08ab694.slice/crio-db2204ac6e07e5f39f15a1240aa33a250894f4c1dc0896058d4301286a12fc12 WatchSource:0}: Error finding container db2204ac6e07e5f39f15a1240aa33a250894f4c1dc0896058d4301286a12fc12: Status 404 returned error can't find the container with id db2204ac6e07e5f39f15a1240aa33a250894f4c1dc0896058d4301286a12fc12
Sep 19 10:38:12.368487 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.368376    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-f6f8d447c-5xpgf"]
Sep 19 10:38:12.371864 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:38:12.371830    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5d7b77b_8c41_4139_b39f_a277f9a4bc54.slice/crio-0cb45019c17ddcc7eee797228fc763b94e8766f0419a4cd52ab18f011285d6dc WatchSource:0}: Error finding container 0cb45019c17ddcc7eee797228fc763b94e8766f0419a4cd52ab18f011285d6dc: Status 404 returned error can't find the container with id 0cb45019c17ddcc7eee797228fc763b94e8766f0419a4cd52ab18f011285d6dc
Sep 19 10:38:12.390839 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.390808    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:12.685351 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.685321    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 19 10:38:12.688568 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:38:12.688529    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2a07f45_a695_43dc_aba0_35579c4ca70b.slice/crio-315f9162987e8a91d5c1b986b94ab873037b4923f76327be251cee6838e8168d WatchSource:0}: Error finding container 315f9162987e8a91d5c1b986b94ab873037b4923f76327be251cee6838e8168d: Status 404 returned error can't find the container with id 315f9162987e8a91d5c1b986b94ab873037b4923f76327be251cee6838e8168d
Sep 19 10:38:12.699481 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.699457    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f","Type":"ContainerStarted","Data":"784a7bf4ef6e3f37667a440fd05a139fba5b2069ecb578d1b247f9453764e642"}
Sep 19 10:38:12.699614 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.699517    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f","Type":"ContainerStarted","Data":"9e2cd6c1ed92bb2d00323798665e3853daf92365dd438caa1000582e36a69fdc"}
Sep 19 10:38:12.699614 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.699543    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f","Type":"ContainerStarted","Data":"f0912466e2d81301dd1aa2e0dd519a217c84d5a5234361d837522caa52223897"}
Sep 19 10:38:12.699614 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.699569    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f","Type":"ContainerStarted","Data":"66d14938b0e4110417e0f7e52695af17e95ce2d46fb86541e81f8027d3a45812"}
Sep 19 10:38:12.706846 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.706812    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh" event={"ID":"5d46d9af-8b3d-4658-bef1-667d76a7d3f2","Type":"ContainerStarted","Data":"ffda823df4fa54a465350b390f5df2b014a7b5165d216567aba5717a0ad71a3d"}
Sep 19 10:38:12.707008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.706916    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh" event={"ID":"5d46d9af-8b3d-4658-bef1-667d76a7d3f2","Type":"ContainerStarted","Data":"7cb74e8c6e47183f387d7a0e52afb7256b9a966311595e046d7ca1c7a332dfd3"}
Sep 19 10:38:12.707008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.706948    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh" event={"ID":"5d46d9af-8b3d-4658-bef1-667d76a7d3f2","Type":"ContainerStarted","Data":"d59c644260e6bd2a1d1f25267efd872c2646f31671878f841f1cf8dcaacb6f8c"}
Sep 19 10:38:12.708184 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.708163    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b2a07f45-a695-43dc-aba0-35579c4ca70b","Type":"ContainerStarted","Data":"315f9162987e8a91d5c1b986b94ab873037b4923f76327be251cee6838e8168d"}
Sep 19 10:38:12.709365 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.709342    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-pk78w" event={"ID":"3bb51bef-73f4-4b3a-9c10-b2eef08ab694","Type":"ContainerStarted","Data":"db2204ac6e07e5f39f15a1240aa33a250894f4c1dc0896058d4301286a12fc12"}
Sep 19 10:38:12.710462 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:12.710443    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf" event={"ID":"a5d7b77b-8c41-4139-b39f-a277f9a4bc54","Type":"ContainerStarted","Data":"0cb45019c17ddcc7eee797228fc763b94e8766f0419a4cd52ab18f011285d6dc"}
Sep 19 10:38:13.720189 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:13.720147    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f","Type":"ContainerStarted","Data":"45cf9b99d4a65f584dcaa99a71782bc52e7cf0fcc44534a63aa55a9d406c2e95"}
Sep 19 10:38:13.722101 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:13.722070    2576 generic.go:356] "Generic (PLEG): container finished" podID="b2a07f45-a695-43dc-aba0-35579c4ca70b" containerID="d61150d67bc7f715354694488151148ae330f8c9a9e19232abb27ee17f185dc5" exitCode=0
Sep 19 10:38:13.722246 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:13.722143    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b2a07f45-a695-43dc-aba0-35579c4ca70b","Type":"ContainerDied","Data":"d61150d67bc7f715354694488151148ae330f8c9a9e19232abb27ee17f185dc5"}
Sep 19 10:38:15.740075 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:15.739945    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f","Type":"ContainerStarted","Data":"2276229cdff2de1a4bbfb4587ad1779b9fce568828b0b4dec2e612ac2e4cb10d"}
Sep 19 10:38:15.743754 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:15.743715    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh" event={"ID":"5d46d9af-8b3d-4658-bef1-667d76a7d3f2","Type":"ContainerStarted","Data":"967b093e1136bde93f71a072ee256da6eec9fdeecb8c69e6d0e197eb5ed58ee4"}
Sep 19 10:38:15.743928 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:15.743809    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh" event={"ID":"5d46d9af-8b3d-4658-bef1-667d76a7d3f2","Type":"ContainerStarted","Data":"e97a6119951066551989395e93d4fa298d6b8ce1af0fb0e647717bc60eb19cf4"}
Sep 19 10:38:15.743928 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:15.743861    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:15.743928 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:15.743901    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh" event={"ID":"5d46d9af-8b3d-4658-bef1-667d76a7d3f2","Type":"ContainerStarted","Data":"8f9152bf63e3c90f402f705e433d666e7a89dfe95f44dabcd09968a4e1d1a972"}
Sep 19 10:38:15.746698 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:15.746655    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-pk78w" event={"ID":"3bb51bef-73f4-4b3a-9c10-b2eef08ab694","Type":"ContainerStarted","Data":"833d18e4109845e897205f56660353c4be58e6c3ae5aa66c48272081d470bf1b"}
Sep 19 10:38:15.746858 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:15.746841    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-pk78w"
Sep 19 10:38:15.749899 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:15.749870    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf" event={"ID":"a5d7b77b-8c41-4139-b39f-a277f9a4bc54","Type":"ContainerStarted","Data":"0a1f8f09a68c5439cad349bc343859d0b9bfd7ff971445e43cfefc1350c9f201"}
Sep 19 10:38:15.750033 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:15.749918    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf" event={"ID":"a5d7b77b-8c41-4139-b39f-a277f9a4bc54","Type":"ContainerStarted","Data":"47d52ba06885a8044fea8acf0a9850183c34f572fbd5d57a12c9f270cf3df86d"}
Sep 19 10:38:15.750033 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:15.749944    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf" event={"ID":"a5d7b77b-8c41-4139-b39f-a277f9a4bc54","Type":"ContainerStarted","Data":"840cb709ad50af69e3a993cd6e0b93124158108554843cc2543c2fec9dcf1d4f"}
Sep 19 10:38:15.753353 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:15.753316    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-pk78w"
Sep 19 10:38:15.765227 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:15.765170    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.335324065 podStartE2EDuration="9.765151913s" totalImagesPullingTime="5.410606344s" totalInitContainerRuntime="19.221504ms" isStatefulPod=false podCreationTimestamp="2026-09-19 10:38:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:38:15.763145293 +0000 UTC m=+105.259451384" watchObservedRunningTime="2026-09-19 10:38:15.765151913 +0000 UTC m=+105.261458004"
Sep 19 10:38:15.779341 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:15.779276    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5984459ff8-pk78w" podStartSLOduration=3.387252495 podStartE2EDuration="5.779254835s" totalImagesPullingTime="2.39200234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:38:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:38:15.777857553 +0000 UTC m=+105.274163647" watchObservedRunningTime="2026-09-19 10:38:15.779254835 +0000 UTC m=+105.275560928"
Sep 19 10:38:15.807020 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:15.806892    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh" podStartSLOduration=2.937624022 podStartE2EDuration="7.806871563s" totalImagesPullingTime="4.869247541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:38:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:38:15.803834172 +0000 UTC m=+105.300140264" watchObservedRunningTime="2026-09-19 10:38:15.806871563 +0000 UTC m=+105.303177656"
Sep 19 10:38:15.836649 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:15.836581    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-f6f8d447c-5xpgf" podStartSLOduration=3.089021989 podStartE2EDuration="5.836557444s" totalImagesPullingTime="2.747535455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:38:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:38:15.834934177 +0000 UTC m=+105.331240270" watchObservedRunningTime="2026-09-19 10:38:15.836557444 +0000 UTC m=+105.332863536"
Sep 19 10:38:17.763554 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:17.763484    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b2a07f45-a695-43dc-aba0-35579c4ca70b","Type":"ContainerStarted","Data":"9de467d18eb2a79524226bdc5cafd97af58a0f6fcf3e0fcf9b51d320c72e3ee6"}
Sep 19 10:38:18.771565 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:18.771521    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b2a07f45-a695-43dc-aba0-35579c4ca70b","Type":"ContainerStarted","Data":"6466bf3e18c304e713e1d44dae898ac742c544bca1dc925997e8780585cc5598"}
Sep 19 10:38:18.772082 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:18.771639    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b2a07f45-a695-43dc-aba0-35579c4ca70b","Type":"ContainerStarted","Data":"978d40d58c7c63a6f2f5f638574d28e8883036bbce85981304582b78cdbb14ba"}
Sep 19 10:38:18.772082 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:18.771720    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b2a07f45-a695-43dc-aba0-35579c4ca70b","Type":"ContainerStarted","Data":"d3dcf99b8969f67d2baa96ced17af55da4033de2f2ff741091a4a063d09311e4"}
Sep 19 10:38:18.772082 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:18.771770    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b2a07f45-a695-43dc-aba0-35579c4ca70b","Type":"ContainerStarted","Data":"a2d9db3d6e2c5dbe08616120b353e02ff85856286a8cdc2f9f67dd85ac3906a7"}
Sep 19 10:38:18.772082 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:18.771816    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b2a07f45-a695-43dc-aba0-35579c4ca70b","Type":"ContainerStarted","Data":"7d814b82e6167cba71d5af6873f5d3aa711b9bfd1bfeff1177863ce8c4af49a8"}
Sep 19 10:38:18.797821 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:18.797738    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.815295826 podStartE2EDuration="6.797714833s" totalImagesPullingTime="3.94710962s" totalInitContainerRuntime="35.309387ms" isStatefulPod=false podCreationTimestamp="2026-09-19 10:38:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:38:18.795859775 +0000 UTC m=+108.292165869" watchObservedRunningTime="2026-09-19 10:38:18.797714833 +0000 UTC m=+108.294020925"
Sep 19 10:38:21.760842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:21.760809    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-79789845bb-wzxrh"
Sep 19 10:38:22.391672 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:22.391636    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:38:26.843793 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:26.843747    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dc6f799bf-9zsks"]
Sep 19 10:38:27.815005 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:27.814969    2576 generic.go:356] "Generic (PLEG): container finished" podID="26cdce24-be14-4ec9-a4cd-403bde09dd5d" containerID="00e4d297de1c6649a2a6d735a0efcf44302c6e37c1ce170fb5ba86990c583a62" exitCode=0
Sep 19 10:38:27.815206 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:27.815070    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-96xsm" event={"ID":"26cdce24-be14-4ec9-a4cd-403bde09dd5d","Type":"ContainerDied","Data":"00e4d297de1c6649a2a6d735a0efcf44302c6e37c1ce170fb5ba86990c583a62"}
Sep 19 10:38:27.815430 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:27.815416    2576 scope.go:117] "RemoveContainer" containerID="00e4d297de1c6649a2a6d735a0efcf44302c6e37c1ce170fb5ba86990c583a62"
Sep 19 10:38:28.821850 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:28.821813    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-96xsm" event={"ID":"26cdce24-be14-4ec9-a4cd-403bde09dd5d","Type":"ContainerStarted","Data":"a62f15532b38cd51f9de89a685ee95fc78a13b5bd46523544ac557dc01fa68bf"}
Sep 19 10:38:40.874329 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:40.874293    2576 generic.go:356] "Generic (PLEG): container finished" podID="2bfbbce9-4493-4ec5-b0d5-d85da7e26e65" containerID="e4125f40b421490e7b11848c882301714af1a43066b073eb26df04eddf95ce75" exitCode=0
Sep 19 10:38:40.874750 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:40.874367    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-44pw4" event={"ID":"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65","Type":"ContainerDied","Data":"e4125f40b421490e7b11848c882301714af1a43066b073eb26df04eddf95ce75"}
Sep 19 10:38:40.874750 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:40.874746    2576 scope.go:117] "RemoveContainer" containerID="e4125f40b421490e7b11848c882301714af1a43066b073eb26df04eddf95ce75"
Sep 19 10:38:41.881399 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:41.881356    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-44pw4" event={"ID":"2bfbbce9-4493-4ec5-b0d5-d85da7e26e65","Type":"ContainerStarted","Data":"55fc5e041c6c5c6266a6cf8c67e588ed5b659d1c12b973066ca7b75c95a9920d"}
Sep 19 10:38:45.899059 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:45.899000    2576 generic.go:356] "Generic (PLEG): container finished" podID="dcf226b4-3c7f-40c0-a327-96003c35e314" containerID="3759b99aecd166bdc282ab56043ba1f1715da8622dcc4767f09cbd7a3f79fd10" exitCode=0
Sep 19 10:38:45.899501 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:45.899076    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6dllh" event={"ID":"dcf226b4-3c7f-40c0-a327-96003c35e314","Type":"ContainerDied","Data":"3759b99aecd166bdc282ab56043ba1f1715da8622dcc4767f09cbd7a3f79fd10"}
Sep 19 10:38:45.899501 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:45.899464    2576 scope.go:117] "RemoveContainer" containerID="3759b99aecd166bdc282ab56043ba1f1715da8622dcc4767f09cbd7a3f79fd10"
Sep 19 10:38:46.907535 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:46.907494    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-6dllh" event={"ID":"dcf226b4-3c7f-40c0-a327-96003c35e314","Type":"ContainerStarted","Data":"2d39be55b1dfac1fd4d17b4ce5bc31b43dd3211f677a961cdb32ce214fa558c3"}
Sep 19 10:38:51.866824 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:51.866756    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7dc6f799bf-9zsks" podUID="4c344464-353e-4403-aa89-2522f7a4a886" containerName="console" containerID="cri-o://4981446a9bba8edc2c8a5f23550b287fb1b28929b1a11e9cef5b6302950bc6e7" gracePeriod=15
Sep 19 10:38:52.165133 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.165091    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:38:52.246279 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.246239    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gt42r\" (UniqueName: \"kubernetes.io/projected/4c344464-353e-4403-aa89-2522f7a4a886-kube-api-access-gt42r\") pod \"4c344464-353e-4403-aa89-2522f7a4a886\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") "
Sep 19 10:38:52.246279 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.246290    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-trusted-ca-bundle\") pod \"4c344464-353e-4403-aa89-2522f7a4a886\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") "
Sep 19 10:38:52.246522 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.246308    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c344464-353e-4403-aa89-2522f7a4a886-console-serving-cert\") pod \"4c344464-353e-4403-aa89-2522f7a4a886\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") "
Sep 19 10:38:52.246522 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.246345    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-service-ca\") pod \"4c344464-353e-4403-aa89-2522f7a4a886\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") "
Sep 19 10:38:52.246522 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.246460    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-oauth-serving-cert\") pod \"4c344464-353e-4403-aa89-2522f7a4a886\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") "
Sep 19 10:38:52.246522 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.246501    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c344464-353e-4403-aa89-2522f7a4a886-console-oauth-config\") pod \"4c344464-353e-4403-aa89-2522f7a4a886\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") "
Sep 19 10:38:52.246738 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.246526    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-console-config\") pod \"4c344464-353e-4403-aa89-2522f7a4a886\" (UID: \"4c344464-353e-4403-aa89-2522f7a4a886\") "
Sep 19 10:38:52.246793 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.246736    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4c344464-353e-4403-aa89-2522f7a4a886" (UID: "4c344464-353e-4403-aa89-2522f7a4a886"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:38:52.246793 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.246777    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-service-ca" (OuterVolumeSpecName: "service-ca") pod "4c344464-353e-4403-aa89-2522f7a4a886" (UID: "4c344464-353e-4403-aa89-2522f7a4a886"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:38:52.246894 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.246861    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4c344464-353e-4403-aa89-2522f7a4a886" (UID: "4c344464-353e-4403-aa89-2522f7a4a886"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:38:52.246894 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.246879    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-trusted-ca-bundle\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:38:52.246993 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.246900    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-service-ca\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:38:52.247125 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.247098    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-console-config" (OuterVolumeSpecName: "console-config") pod "4c344464-353e-4403-aa89-2522f7a4a886" (UID: "4c344464-353e-4403-aa89-2522f7a4a886"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:38:52.248940 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.248913    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c344464-353e-4403-aa89-2522f7a4a886-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4c344464-353e-4403-aa89-2522f7a4a886" (UID: "4c344464-353e-4403-aa89-2522f7a4a886"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:38:52.249090 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.248941    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c344464-353e-4403-aa89-2522f7a4a886-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4c344464-353e-4403-aa89-2522f7a4a886" (UID: "4c344464-353e-4403-aa89-2522f7a4a886"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:38:52.249090 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.249068    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c344464-353e-4403-aa89-2522f7a4a886-kube-api-access-gt42r" (OuterVolumeSpecName: "kube-api-access-gt42r") pod "4c344464-353e-4403-aa89-2522f7a4a886" (UID: "4c344464-353e-4403-aa89-2522f7a4a886"). InnerVolumeSpecName "kube-api-access-gt42r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:38:52.347612 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.347570    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-oauth-serving-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:38:52.347612 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.347601    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c344464-353e-4403-aa89-2522f7a4a886-console-oauth-config\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:38:52.347612 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.347613    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c344464-353e-4403-aa89-2522f7a4a886-console-config\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:38:52.347895 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.347625    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gt42r\" (UniqueName: \"kubernetes.io/projected/4c344464-353e-4403-aa89-2522f7a4a886-kube-api-access-gt42r\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:38:52.347895 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.347640    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c344464-353e-4403-aa89-2522f7a4a886-console-serving-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:38:52.936747 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.936694    2576 generic.go:356] "Generic (PLEG): container finished" podID="4c344464-353e-4403-aa89-2522f7a4a886" containerID="4981446a9bba8edc2c8a5f23550b287fb1b28929b1a11e9cef5b6302950bc6e7" exitCode=0
Sep 19 10:38:52.937216 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.936807    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dc6f799bf-9zsks"
Sep 19 10:38:52.937216 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.936835    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc6f799bf-9zsks" event={"ID":"4c344464-353e-4403-aa89-2522f7a4a886","Type":"ContainerDied","Data":"4981446a9bba8edc2c8a5f23550b287fb1b28929b1a11e9cef5b6302950bc6e7"}
Sep 19 10:38:52.937216 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.936888    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dc6f799bf-9zsks" event={"ID":"4c344464-353e-4403-aa89-2522f7a4a886","Type":"ContainerDied","Data":"8f776a1d4377fa90d800ee56e02a3fa313d022a462ea7fab4e3b12151eb4cb3b"}
Sep 19 10:38:52.937216 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.936909    2576 scope.go:117] "RemoveContainer" containerID="4981446a9bba8edc2c8a5f23550b287fb1b28929b1a11e9cef5b6302950bc6e7"
Sep 19 10:38:52.949519 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.948245    2576 scope.go:117] "RemoveContainer" containerID="4981446a9bba8edc2c8a5f23550b287fb1b28929b1a11e9cef5b6302950bc6e7"
Sep 19 10:38:52.949519 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:38:52.949325    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4981446a9bba8edc2c8a5f23550b287fb1b28929b1a11e9cef5b6302950bc6e7\": container with ID starting with 4981446a9bba8edc2c8a5f23550b287fb1b28929b1a11e9cef5b6302950bc6e7 not found: ID does not exist" containerID="4981446a9bba8edc2c8a5f23550b287fb1b28929b1a11e9cef5b6302950bc6e7"
Sep 19 10:38:52.949519 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.949427    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4981446a9bba8edc2c8a5f23550b287fb1b28929b1a11e9cef5b6302950bc6e7"} err="failed to get container status \"4981446a9bba8edc2c8a5f23550b287fb1b28929b1a11e9cef5b6302950bc6e7\": rpc error: code = NotFound desc = could not find container \"4981446a9bba8edc2c8a5f23550b287fb1b28929b1a11e9cef5b6302950bc6e7\": container with ID starting with 4981446a9bba8edc2c8a5f23550b287fb1b28929b1a11e9cef5b6302950bc6e7 not found: ID does not exist"
Sep 19 10:38:52.965434 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.965403    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dc6f799bf-9zsks"]
Sep 19 10:38:52.971351 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:52.971318    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7dc6f799bf-9zsks"]
Sep 19 10:38:53.000671 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:38:53.000635    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c344464-353e-4403-aa89-2522f7a4a886" path="/var/lib/kubelet/pods/4c344464-353e-4403-aa89-2522f7a4a886/volumes"
Sep 19 10:39:12.391529 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:39:12.391473    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:39:12.408877 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:39:12.408842    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:39:13.049816 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:39:13.049781    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 19 10:41:30.883190 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:30.883154    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 10:41:30.886194 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:30.886168    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 10:41:30.893807 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:30.893781    2576 kubelet.go:1648] "Image garbage collection succeeded"
Sep 19 10:41:48.304930 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.304890    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"]
Sep 19 10:41:48.307472 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.305292    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c344464-353e-4403-aa89-2522f7a4a886" containerName="console"
Sep 19 10:41:48.307472 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.305305    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c344464-353e-4403-aa89-2522f7a4a886" containerName="console"
Sep 19 10:41:48.307472 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.305362    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c344464-353e-4403-aa89-2522f7a4a886" containerName="console"
Sep 19 10:41:48.308459 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.308443    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"
Sep 19 10:41:48.311108 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.311067    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 19 10:41:48.311335 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.311306    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zrzqf\""
Sep 19 10:41:48.311335 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.311331    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 19 10:41:48.311493 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.311302    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 19 10:41:48.311493 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.311390    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 19 10:41:48.326078 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.326028    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"]
Sep 19 10:41:48.379144 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.379094    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm2p4\" (UniqueName: \"kubernetes.io/projected/42161c4c-6456-45dc-a90f-8623b3a80457-kube-api-access-xm2p4\") pod \"opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x\" (UID: \"42161c4c-6456-45dc-a90f-8623b3a80457\") " pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"
Sep 19 10:41:48.379352 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.379186    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/42161c4c-6456-45dc-a90f-8623b3a80457-manifests\") pod \"opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x\" (UID: \"42161c4c-6456-45dc-a90f-8623b3a80457\") " pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"
Sep 19 10:41:48.379352 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.379232    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42161c4c-6456-45dc-a90f-8623b3a80457-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x\" (UID: \"42161c4c-6456-45dc-a90f-8623b3a80457\") " pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"
Sep 19 10:41:48.379352 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.379289    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42161c4c-6456-45dc-a90f-8623b3a80457-tmp\") pod \"opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x\" (UID: \"42161c4c-6456-45dc-a90f-8623b3a80457\") " pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"
Sep 19 10:41:48.379352 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.379323    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/42161c4c-6456-45dc-a90f-8623b3a80457-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x\" (UID: \"42161c4c-6456-45dc-a90f-8623b3a80457\") " pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"
Sep 19 10:41:48.480333 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.480292    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42161c4c-6456-45dc-a90f-8623b3a80457-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x\" (UID: \"42161c4c-6456-45dc-a90f-8623b3a80457\") " pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"
Sep 19 10:41:48.480499 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.480359    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42161c4c-6456-45dc-a90f-8623b3a80457-tmp\") pod \"opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x\" (UID: \"42161c4c-6456-45dc-a90f-8623b3a80457\") " pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"
Sep 19 10:41:48.480499 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.480396    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/42161c4c-6456-45dc-a90f-8623b3a80457-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x\" (UID: \"42161c4c-6456-45dc-a90f-8623b3a80457\") " pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"
Sep 19 10:41:48.480499 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.480432    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm2p4\" (UniqueName: \"kubernetes.io/projected/42161c4c-6456-45dc-a90f-8623b3a80457-kube-api-access-xm2p4\") pod \"opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x\" (UID: \"42161c4c-6456-45dc-a90f-8623b3a80457\") " pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"
Sep 19 10:41:48.480604 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.480499    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/42161c4c-6456-45dc-a90f-8623b3a80457-manifests\") pod \"opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x\" (UID: \"42161c4c-6456-45dc-a90f-8623b3a80457\") " pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"
Sep 19 10:41:48.480904 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.480884    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/42161c4c-6456-45dc-a90f-8623b3a80457-manifests\") pod \"opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x\" (UID: \"42161c4c-6456-45dc-a90f-8623b3a80457\") " pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"
Sep 19 10:41:48.483162 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.483141    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42161c4c-6456-45dc-a90f-8623b3a80457-tmp\") pod \"opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x\" (UID: \"42161c4c-6456-45dc-a90f-8623b3a80457\") " pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"
Sep 19 10:41:48.483249 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.483209    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42161c4c-6456-45dc-a90f-8623b3a80457-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x\" (UID: \"42161c4c-6456-45dc-a90f-8623b3a80457\") " pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"
Sep 19 10:41:48.483337 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.483323    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/42161c4c-6456-45dc-a90f-8623b3a80457-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x\" (UID: \"42161c4c-6456-45dc-a90f-8623b3a80457\") " pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"
Sep 19 10:41:48.492149 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.492122    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm2p4\" (UniqueName: \"kubernetes.io/projected/42161c4c-6456-45dc-a90f-8623b3a80457-kube-api-access-xm2p4\") pod \"opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x\" (UID: \"42161c4c-6456-45dc-a90f-8623b3a80457\") " pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"
Sep 19 10:41:48.619683 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.619641    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"
Sep 19 10:41:48.904093 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.903990    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"]
Sep 19 10:41:48.908681 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:41:48.908647    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42161c4c_6456_45dc_a90f_8623b3a80457.slice/crio-d5baf49649c94c9cf46035442d447e0c9b90b80bdd44cfe97c37d711ccfea793 WatchSource:0}: Error finding container d5baf49649c94c9cf46035442d447e0c9b90b80bdd44cfe97c37d711ccfea793: Status 404 returned error can't find the container with id d5baf49649c94c9cf46035442d447e0c9b90b80bdd44cfe97c37d711ccfea793
Sep 19 10:41:48.910673 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:48.910652    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 10:41:49.713342 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:49.713302    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x" event={"ID":"42161c4c-6456-45dc-a90f-8623b3a80457","Type":"ContainerStarted","Data":"d5baf49649c94c9cf46035442d447e0c9b90b80bdd44cfe97c37d711ccfea793"}
Sep 19 10:41:52.729209 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:52.729170    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x" event={"ID":"42161c4c-6456-45dc-a90f-8623b3a80457","Type":"ContainerStarted","Data":"3fcc60917a4e513db212f93bed8ba86d1a7ec4418754b3ac110ab23ebad7cdb5"}
Sep 19 10:41:53.735301 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:53.735254    2576 generic.go:356] "Generic (PLEG): container finished" podID="42161c4c-6456-45dc-a90f-8623b3a80457" containerID="3fcc60917a4e513db212f93bed8ba86d1a7ec4418754b3ac110ab23ebad7cdb5" exitCode=0
Sep 19 10:41:53.735916 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:53.735308    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x" event={"ID":"42161c4c-6456-45dc-a90f-8623b3a80457","Type":"ContainerDied","Data":"3fcc60917a4e513db212f93bed8ba86d1a7ec4418754b3ac110ab23ebad7cdb5"}
Sep 19 10:41:54.741732 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:54.741695    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x" event={"ID":"42161c4c-6456-45dc-a90f-8623b3a80457","Type":"ContainerStarted","Data":"1c94b27d86cbc0a14447e81a94e1d1f72cf998af1d44878fa3885c603547d6d1"}
Sep 19 10:41:54.742166 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:54.741870    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"
Sep 19 10:41:54.764804 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:41:54.764730    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x" podStartSLOduration=2.352806673 podStartE2EDuration="6.764706864s" totalImagesPullingTime="3.247298739s" totalInitContainerRuntime="1.164601452s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:41:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:41:54.760868657 +0000 UTC m=+324.257174750" watchObservedRunningTime="2026-09-19 10:41:54.764706864 +0000 UTC m=+324.261012956"
Sep 19 10:42:05.748500 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:05.748472    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x"
Sep 19 10:42:07.796758 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:07.796713    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt"]
Sep 19 10:42:07.806642 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:07.806617    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt"
Sep 19 10:42:07.809244 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:07.809201    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 19 10:42:07.809244 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:07.809224    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 19 10:42:07.809244 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:07.809235    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-sm98g\""
Sep 19 10:42:07.809507 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:07.809339    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 19 10:42:07.809507 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:07.809379    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 19 10:42:07.809591 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:07.809577    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 19 10:42:07.812739 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:07.812717    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt"]
Sep 19 10:42:07.962057 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:07.962012    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsh42\" (UniqueName: \"kubernetes.io/projected/00720b60-9494-4b3a-a68e-7ae372dba963-kube-api-access-gsh42\") pod \"lws-controller-manager-749b85b68c-fl7vt\" (UID: \"00720b60-9494-4b3a-a68e-7ae372dba963\") " pod="openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt"
Sep 19 10:42:07.962251 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:07.962140    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00720b60-9494-4b3a-a68e-7ae372dba963-cert\") pod \"lws-controller-manager-749b85b68c-fl7vt\" (UID: \"00720b60-9494-4b3a-a68e-7ae372dba963\") " pod="openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt"
Sep 19 10:42:07.962251 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:07.962185    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/00720b60-9494-4b3a-a68e-7ae372dba963-manager-config\") pod \"lws-controller-manager-749b85b68c-fl7vt\" (UID: \"00720b60-9494-4b3a-a68e-7ae372dba963\") " pod="openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt"
Sep 19 10:42:07.962251 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:07.962238    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/00720b60-9494-4b3a-a68e-7ae372dba963-metrics-cert\") pod \"lws-controller-manager-749b85b68c-fl7vt\" (UID: \"00720b60-9494-4b3a-a68e-7ae372dba963\") " pod="openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt"
Sep 19 10:42:08.063679 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:08.063592    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/00720b60-9494-4b3a-a68e-7ae372dba963-manager-config\") pod \"lws-controller-manager-749b85b68c-fl7vt\" (UID: \"00720b60-9494-4b3a-a68e-7ae372dba963\") " pod="openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt"
Sep 19 10:42:08.063679 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:08.063649    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/00720b60-9494-4b3a-a68e-7ae372dba963-metrics-cert\") pod \"lws-controller-manager-749b85b68c-fl7vt\" (UID: \"00720b60-9494-4b3a-a68e-7ae372dba963\") " pod="openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt"
Sep 19 10:42:08.063920 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:08.063690    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsh42\" (UniqueName: \"kubernetes.io/projected/00720b60-9494-4b3a-a68e-7ae372dba963-kube-api-access-gsh42\") pod \"lws-controller-manager-749b85b68c-fl7vt\" (UID: \"00720b60-9494-4b3a-a68e-7ae372dba963\") " pod="openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt"
Sep 19 10:42:08.063920 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:08.063719    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00720b60-9494-4b3a-a68e-7ae372dba963-cert\") pod \"lws-controller-manager-749b85b68c-fl7vt\" (UID: \"00720b60-9494-4b3a-a68e-7ae372dba963\") " pod="openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt"
Sep 19 10:42:08.064299 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:08.064278    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/00720b60-9494-4b3a-a68e-7ae372dba963-manager-config\") pod \"lws-controller-manager-749b85b68c-fl7vt\" (UID: \"00720b60-9494-4b3a-a68e-7ae372dba963\") " pod="openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt"
Sep 19 10:42:08.066366 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:08.066339    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/00720b60-9494-4b3a-a68e-7ae372dba963-metrics-cert\") pod \"lws-controller-manager-749b85b68c-fl7vt\" (UID: \"00720b60-9494-4b3a-a68e-7ae372dba963\") " pod="openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt"
Sep 19 10:42:08.066472 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:08.066455    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00720b60-9494-4b3a-a68e-7ae372dba963-cert\") pod \"lws-controller-manager-749b85b68c-fl7vt\" (UID: \"00720b60-9494-4b3a-a68e-7ae372dba963\") " pod="openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt"
Sep 19 10:42:08.071496 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:08.071471    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsh42\" (UniqueName: \"kubernetes.io/projected/00720b60-9494-4b3a-a68e-7ae372dba963-kube-api-access-gsh42\") pod \"lws-controller-manager-749b85b68c-fl7vt\" (UID: \"00720b60-9494-4b3a-a68e-7ae372dba963\") " pod="openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt"
Sep 19 10:42:08.117709 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:08.117672    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt"
Sep 19 10:42:08.392305 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:08.392280    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt"]
Sep 19 10:42:08.394455 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:42:08.394423    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00720b60_9494_4b3a_a68e_7ae372dba963.slice/crio-15a5ccaa3fb88002474295639a901de92d70dd15bf2eb82569f6723938a52a1b WatchSource:0}: Error finding container 15a5ccaa3fb88002474295639a901de92d70dd15bf2eb82569f6723938a52a1b: Status 404 returned error can't find the container with id 15a5ccaa3fb88002474295639a901de92d70dd15bf2eb82569f6723938a52a1b
Sep 19 10:42:08.807781 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:08.807691    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt" event={"ID":"00720b60-9494-4b3a-a68e-7ae372dba963","Type":"ContainerStarted","Data":"15a5ccaa3fb88002474295639a901de92d70dd15bf2eb82569f6723938a52a1b"}
Sep 19 10:42:12.830503 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:12.830468    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt" event={"ID":"00720b60-9494-4b3a-a68e-7ae372dba963","Type":"ContainerStarted","Data":"27c3b2ea7984df1ee4058d622bcd237a8f3c40922373a88ac948414820280c0a"}
Sep 19 10:42:12.830894 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:12.830606    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt"
Sep 19 10:42:12.847201 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:12.847151    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt" podStartSLOduration=2.077816704 podStartE2EDuration="5.847136512s" totalImagesPullingTime="3.769319808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:42:12.844557268 +0000 UTC m=+342.340863362" watchObservedRunningTime="2026-09-19 10:42:12.847136512 +0000 UTC m=+342.343442604"
Sep 19 10:42:17.845651 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:17.845612    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6d97bd96f8-cjbpx"]
Sep 19 10:42:17.849401 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:17.849380    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6d97bd96f8-cjbpx"
Sep 19 10:42:17.852954 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:17.852919    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 19 10:42:17.852954 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:17.852919    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-rp7xj\""
Sep 19 10:42:17.853169 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:17.852919    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 19 10:42:17.853169 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:17.853113    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fb08888-aa5a-4e4d-a9ba-e96f011060f7-tls-certs\") pod \"kube-auth-proxy-6d97bd96f8-cjbpx\" (UID: \"6fb08888-aa5a-4e4d-a9ba-e96f011060f7\") " pod="openshift-ingress/kube-auth-proxy-6d97bd96f8-cjbpx"
Sep 19 10:42:17.853251 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:17.853196    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbk9m\" (UniqueName: \"kubernetes.io/projected/6fb08888-aa5a-4e4d-a9ba-e96f011060f7-kube-api-access-tbk9m\") pod \"kube-auth-proxy-6d97bd96f8-cjbpx\" (UID: \"6fb08888-aa5a-4e4d-a9ba-e96f011060f7\") " pod="openshift-ingress/kube-auth-proxy-6d97bd96f8-cjbpx"
Sep 19 10:42:17.853302 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:17.853224    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fb08888-aa5a-4e4d-a9ba-e96f011060f7-tmp\") pod \"kube-auth-proxy-6d97bd96f8-cjbpx\" (UID: \"6fb08888-aa5a-4e4d-a9ba-e96f011060f7\") " pod="openshift-ingress/kube-auth-proxy-6d97bd96f8-cjbpx"
Sep 19 10:42:17.857654 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:17.857629    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6d97bd96f8-cjbpx"]
Sep 19 10:42:17.953913 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:17.953880    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fb08888-aa5a-4e4d-a9ba-e96f011060f7-tls-certs\") pod \"kube-auth-proxy-6d97bd96f8-cjbpx\" (UID: \"6fb08888-aa5a-4e4d-a9ba-e96f011060f7\") " pod="openshift-ingress/kube-auth-proxy-6d97bd96f8-cjbpx"
Sep 19 10:42:17.954095 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:17.953947    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbk9m\" (UniqueName: \"kubernetes.io/projected/6fb08888-aa5a-4e4d-a9ba-e96f011060f7-kube-api-access-tbk9m\") pod \"kube-auth-proxy-6d97bd96f8-cjbpx\" (UID: \"6fb08888-aa5a-4e4d-a9ba-e96f011060f7\") " pod="openshift-ingress/kube-auth-proxy-6d97bd96f8-cjbpx"
Sep 19 10:42:17.954095 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:17.953981    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fb08888-aa5a-4e4d-a9ba-e96f011060f7-tmp\") pod \"kube-auth-proxy-6d97bd96f8-cjbpx\" (UID: \"6fb08888-aa5a-4e4d-a9ba-e96f011060f7\") " pod="openshift-ingress/kube-auth-proxy-6d97bd96f8-cjbpx"
Sep 19 10:42:17.956454 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:17.956423    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fb08888-aa5a-4e4d-a9ba-e96f011060f7-tmp\") pod \"kube-auth-proxy-6d97bd96f8-cjbpx\" (UID: \"6fb08888-aa5a-4e4d-a9ba-e96f011060f7\") " pod="openshift-ingress/kube-auth-proxy-6d97bd96f8-cjbpx"
Sep 19 10:42:17.956574 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:17.956515    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fb08888-aa5a-4e4d-a9ba-e96f011060f7-tls-certs\") pod \"kube-auth-proxy-6d97bd96f8-cjbpx\" (UID: \"6fb08888-aa5a-4e4d-a9ba-e96f011060f7\") " pod="openshift-ingress/kube-auth-proxy-6d97bd96f8-cjbpx"
Sep 19 10:42:17.967362 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:17.967335    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbk9m\" (UniqueName: \"kubernetes.io/projected/6fb08888-aa5a-4e4d-a9ba-e96f011060f7-kube-api-access-tbk9m\") pod \"kube-auth-proxy-6d97bd96f8-cjbpx\" (UID: \"6fb08888-aa5a-4e4d-a9ba-e96f011060f7\") " pod="openshift-ingress/kube-auth-proxy-6d97bd96f8-cjbpx"
Sep 19 10:42:18.161834 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:18.161747    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6d97bd96f8-cjbpx"
Sep 19 10:42:18.440897 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:18.440871    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6d97bd96f8-cjbpx"]
Sep 19 10:42:18.443640 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:42:18.443613    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb08888_aa5a_4e4d_a9ba_e96f011060f7.slice/crio-0bce4ca260b66ee5d200e1cd74272e2ed17ae24561fefa253e3b92f6922beede WatchSource:0}: Error finding container 0bce4ca260b66ee5d200e1cd74272e2ed17ae24561fefa253e3b92f6922beede: Status 404 returned error can't find the container with id 0bce4ca260b66ee5d200e1cd74272e2ed17ae24561fefa253e3b92f6922beede
Sep 19 10:42:18.861896 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:18.861865    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6d97bd96f8-cjbpx" event={"ID":"6fb08888-aa5a-4e4d-a9ba-e96f011060f7","Type":"ContainerStarted","Data":"0bce4ca260b66ee5d200e1cd74272e2ed17ae24561fefa253e3b92f6922beede"}
Sep 19 10:42:21.880444 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:21.880362    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6d97bd96f8-cjbpx" event={"ID":"6fb08888-aa5a-4e4d-a9ba-e96f011060f7","Type":"ContainerStarted","Data":"39c8fea0becb80868382f944ae1aa79d437e0a800f88237f9d60dd4eac764884"}
Sep 19 10:42:21.895987 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:21.895937    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6d97bd96f8-cjbpx" podStartSLOduration=2.254180317 podStartE2EDuration="4.895920805s" totalImagesPullingTime="2.641740488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:42:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:42:21.894481786 +0000 UTC m=+351.390787879" watchObservedRunningTime="2026-09-19 10:42:21.895920805 +0000 UTC m=+351.392226897"
Sep 19 10:42:23.837593 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:23.837561    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-749b85b68c-fl7vt"
Sep 19 10:42:46.778881 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.778848    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69ccf88565-8cdtz"]
Sep 19 10:42:46.788334 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.788306    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.790812 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.790789    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 19 10:42:46.793009 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.792174    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 19 10:42:46.793009 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.792443    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 19 10:42:46.793009 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.792479    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 19 10:42:46.793009 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.792615    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-f4tr5\""
Sep 19 10:42:46.793009 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.792711    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 19 10:42:46.793009 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.792778    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 19 10:42:46.793009 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.792913    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 19 10:42:46.795520 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.795093    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69ccf88565-8cdtz"]
Sep 19 10:42:46.797929 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.797764    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 19 10:42:46.893190 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.893160    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ddb962d-a253-4804-a802-a8ec8276ce3f-oauth-serving-cert\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.893341 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.893201    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpqhn\" (UniqueName: \"kubernetes.io/projected/6ddb962d-a253-4804-a802-a8ec8276ce3f-kube-api-access-mpqhn\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.893341 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.893286    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ddb962d-a253-4804-a802-a8ec8276ce3f-service-ca\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.893341 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.893318    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ddb962d-a253-4804-a802-a8ec8276ce3f-console-oauth-config\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.893341 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.893338    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ddb962d-a253-4804-a802-a8ec8276ce3f-console-serving-cert\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.893483 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.893351    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ddb962d-a253-4804-a802-a8ec8276ce3f-console-config\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.893483 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.893366    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ddb962d-a253-4804-a802-a8ec8276ce3f-trusted-ca-bundle\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.994367 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.994341    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ddb962d-a253-4804-a802-a8ec8276ce3f-service-ca\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.994367 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.994374    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ddb962d-a253-4804-a802-a8ec8276ce3f-console-oauth-config\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.994554 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.994394    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ddb962d-a253-4804-a802-a8ec8276ce3f-console-serving-cert\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.994554 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.994504    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ddb962d-a253-4804-a802-a8ec8276ce3f-console-config\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.994554 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.994533    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ddb962d-a253-4804-a802-a8ec8276ce3f-trusted-ca-bundle\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.994670 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.994605    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ddb962d-a253-4804-a802-a8ec8276ce3f-oauth-serving-cert\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.994670 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.994636    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpqhn\" (UniqueName: \"kubernetes.io/projected/6ddb962d-a253-4804-a802-a8ec8276ce3f-kube-api-access-mpqhn\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.995218 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.995178    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ddb962d-a253-4804-a802-a8ec8276ce3f-service-ca\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.995351 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.995189    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ddb962d-a253-4804-a802-a8ec8276ce3f-console-config\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.995405 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.995387    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ddb962d-a253-4804-a802-a8ec8276ce3f-trusted-ca-bundle\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.995582 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.995557    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ddb962d-a253-4804-a802-a8ec8276ce3f-oauth-serving-cert\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.997108 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.997086    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ddb962d-a253-4804-a802-a8ec8276ce3f-console-serving-cert\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:46.997489 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:46.997470    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ddb962d-a253-4804-a802-a8ec8276ce3f-console-oauth-config\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:47.003098 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:47.003034    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpqhn\" (UniqueName: \"kubernetes.io/projected/6ddb962d-a253-4804-a802-a8ec8276ce3f-kube-api-access-mpqhn\") pod \"console-69ccf88565-8cdtz\" (UID: \"6ddb962d-a253-4804-a802-a8ec8276ce3f\") " pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:47.102798 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:47.102771    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:47.385455 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:47.385427    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69ccf88565-8cdtz"]
Sep 19 10:42:47.387669 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:42:47.387626    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ddb962d_a253_4804_a802_a8ec8276ce3f.slice/crio-ed205b060bc21ad39e63b934d412131f48b77f78da8ea032ef2d1f5f944b86cc WatchSource:0}: Error finding container ed205b060bc21ad39e63b934d412131f48b77f78da8ea032ef2d1f5f944b86cc: Status 404 returned error can't find the container with id ed205b060bc21ad39e63b934d412131f48b77f78da8ea032ef2d1f5f944b86cc
Sep 19 10:42:47.998585 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:47.998549    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69ccf88565-8cdtz" event={"ID":"6ddb962d-a253-4804-a802-a8ec8276ce3f","Type":"ContainerStarted","Data":"907b8750b6e32773a7084513078d888607c76814794083ddac1ee8ff6483cd48"}
Sep 19 10:42:47.999036 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:47.998609    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69ccf88565-8cdtz" event={"ID":"6ddb962d-a253-4804-a802-a8ec8276ce3f","Type":"ContainerStarted","Data":"ed205b060bc21ad39e63b934d412131f48b77f78da8ea032ef2d1f5f944b86cc"}
Sep 19 10:42:48.016987 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:48.016929    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69ccf88565-8cdtz" podStartSLOduration=2.016913387 podStartE2EDuration="2.016913387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:42:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:42:48.01616643 +0000 UTC m=+377.512472547" watchObservedRunningTime="2026-09-19 10:42:48.016913387 +0000 UTC m=+377.513219480"
Sep 19 10:42:57.103623 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:57.103588    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:57.103623 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:57.103629    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:57.108572 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:57.108550    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:42:58.048822 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:42:58.048790    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69ccf88565-8cdtz"
Sep 19 10:44:00.770240 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:00.770145    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gk4r6"]
Sep 19 10:44:00.776226 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:00.776192    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gk4r6"
Sep 19 10:44:00.779815 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:00.779796    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 19 10:44:00.779815 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:00.779808    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-cxgff\""
Sep 19 10:44:00.780368 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:00.780347    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 19 10:44:00.791618 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:00.791594    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gk4r6"]
Sep 19 10:44:00.876628 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:00.876588    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-gk4r6\" (UID: \"2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gk4r6"
Sep 19 10:44:00.876781 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:00.876666    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5695f\" (UniqueName: \"kubernetes.io/projected/2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee-kube-api-access-5695f\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-gk4r6\" (UID: \"2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gk4r6"
Sep 19 10:44:00.977301 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:00.977264    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-gk4r6\" (UID: \"2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gk4r6"
Sep 19 10:44:00.977405 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:00.977339    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5695f\" (UniqueName: \"kubernetes.io/projected/2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee-kube-api-access-5695f\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-gk4r6\" (UID: \"2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gk4r6"
Sep 19 10:44:00.977647 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:00.977628    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-gk4r6\" (UID: \"2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gk4r6"
Sep 19 10:44:00.989830 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:00.989803    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5695f\" (UniqueName: \"kubernetes.io/projected/2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee-kube-api-access-5695f\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-gk4r6\" (UID: \"2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gk4r6"
Sep 19 10:44:01.087097 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:01.087075    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gk4r6"
Sep 19 10:44:01.366178 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:01.366155    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gk4r6"]
Sep 19 10:44:01.367600 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:44:01.367571    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bbe01ff_81c4_4a5b_b2c1_aa05316d12ee.slice/crio-5e39bfaf7d6b20750e0f04aa8637bcbd63f21b60fb1c88bb7bba6feae5eac3c1 WatchSource:0}: Error finding container 5e39bfaf7d6b20750e0f04aa8637bcbd63f21b60fb1c88bb7bba6feae5eac3c1: Status 404 returned error can't find the container with id 5e39bfaf7d6b20750e0f04aa8637bcbd63f21b60fb1c88bb7bba6feae5eac3c1
Sep 19 10:44:01.783686 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:01.783605    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gk4r6"]
Sep 19 10:44:01.787149 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:01.787126    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gk4r6"]
Sep 19 10:44:01.807851 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:01.807823    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g"]
Sep 19 10:44:01.812771 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:01.812750    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g"
Sep 19 10:44:01.830943 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:01.830911    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g"]
Sep 19 10:44:01.885281 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:01.885250    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/068969ba-6b70-4cc0-a0bd-c0fc4601ee1f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-mgh2g\" (UID: \"068969ba-6b70-4cc0-a0bd-c0fc4601ee1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g"
Sep 19 10:44:01.885452 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:01.885335    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nw27\" (UniqueName: \"kubernetes.io/projected/068969ba-6b70-4cc0-a0bd-c0fc4601ee1f-kube-api-access-8nw27\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-mgh2g\" (UID: \"068969ba-6b70-4cc0-a0bd-c0fc4601ee1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g"
Sep 19 10:44:01.986085 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:01.986036    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nw27\" (UniqueName: \"kubernetes.io/projected/068969ba-6b70-4cc0-a0bd-c0fc4601ee1f-kube-api-access-8nw27\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-mgh2g\" (UID: \"068969ba-6b70-4cc0-a0bd-c0fc4601ee1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g"
Sep 19 10:44:01.986258 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:01.986168    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/068969ba-6b70-4cc0-a0bd-c0fc4601ee1f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-mgh2g\" (UID: \"068969ba-6b70-4cc0-a0bd-c0fc4601ee1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g"
Sep 19 10:44:01.986588 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:01.986561    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/068969ba-6b70-4cc0-a0bd-c0fc4601ee1f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-mgh2g\" (UID: \"068969ba-6b70-4cc0-a0bd-c0fc4601ee1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g"
Sep 19 10:44:01.994854 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:01.994826    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nw27\" (UniqueName: \"kubernetes.io/projected/068969ba-6b70-4cc0-a0bd-c0fc4601ee1f-kube-api-access-8nw27\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-mgh2g\" (UID: \"068969ba-6b70-4cc0-a0bd-c0fc4601ee1f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g"
Sep 19 10:44:02.124261 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:02.124230    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g"
Sep 19 10:44:02.420424 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:02.420396    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g"]
Sep 19 10:44:02.422792 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:44:02.422753    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod068969ba_6b70_4cc0_a0bd_c0fc4601ee1f.slice/crio-0494bb8515ffa44493e81a9034b3672b99c904ecad4695117bd27eeac8fd2c9e WatchSource:0}: Error finding container 0494bb8515ffa44493e81a9034b3672b99c904ecad4695117bd27eeac8fd2c9e: Status 404 returned error can't find the container with id 0494bb8515ffa44493e81a9034b3672b99c904ecad4695117bd27eeac8fd2c9e
Sep 19 10:44:03.353497 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:03.353456    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g" event={"ID":"068969ba-6b70-4cc0-a0bd-c0fc4601ee1f","Type":"ContainerStarted","Data":"0494bb8515ffa44493e81a9034b3672b99c904ecad4695117bd27eeac8fd2c9e"}
Sep 19 10:44:06.380494 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:06.380453    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g" event={"ID":"068969ba-6b70-4cc0-a0bd-c0fc4601ee1f","Type":"ContainerStarted","Data":"17af5bfe115f5e4b295d623f8647f4d99f412ae8d246f5d1c6acbf9ead3e3ae8"}
Sep 19 10:44:06.380923 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:06.380677    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g"
Sep 19 10:44:06.382336 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:06.382313    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gk4r6" podUID="2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee" containerName="manager" containerID="cri-o://4866fe2ba15ac7de070e59c2599bea7bee2f6335bfbb3e65c012ba43322db876" gracePeriod=2
Sep 19 10:44:06.400642 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:06.400599    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g" podStartSLOduration=2.439810771 podStartE2EDuration="5.400581382s" totalImagesPullingTime="2.960770611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:44:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:44:06.398840645 +0000 UTC m=+455.895146737" watchObservedRunningTime="2026-09-19 10:44:06.400581382 +0000 UTC m=+455.896887476"
Sep 19 10:44:06.619511 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:06.619487    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gk4r6"
Sep 19 10:44:06.621902 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:44:06.621874    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-gk4r6\" is forbidden: User \"system:node:ip-10-0-142-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-165.ec2.internal' and this object" podUID="2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gk4r6"
Sep 19 10:44:06.734474 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:06.734411    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee-extensions-socket-volume\") pod \"2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee\" (UID: \"2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee\") "
Sep 19 10:44:06.734474 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:06.734472    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5695f\" (UniqueName: \"kubernetes.io/projected/2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee-kube-api-access-5695f\") pod \"2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee\" (UID: \"2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee\") "
Sep 19 10:44:06.734662 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:06.734636    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee" (UID: "2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:44:06.734790 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:06.734774    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee-extensions-socket-volume\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:44:06.736774 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:06.736754    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee-kube-api-access-5695f" (OuterVolumeSpecName: "kube-api-access-5695f") pod "2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee" (UID: "2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee"). InnerVolumeSpecName "kube-api-access-5695f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:44:06.835912 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:06.835886    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5695f\" (UniqueName: \"kubernetes.io/projected/2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee-kube-api-access-5695f\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:44:07.001640 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:07.001578    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee" path="/var/lib/kubelet/pods/2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee/volumes"
Sep 19 10:44:07.387877 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:07.387847    2576 generic.go:356] "Generic (PLEG): container finished" podID="2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee" containerID="4866fe2ba15ac7de070e59c2599bea7bee2f6335bfbb3e65c012ba43322db876" exitCode=2
Sep 19 10:44:07.388314 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:07.387892    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gk4r6"
Sep 19 10:44:07.388314 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:07.387910    2576 scope.go:117] "RemoveContainer" containerID="4866fe2ba15ac7de070e59c2599bea7bee2f6335bfbb3e65c012ba43322db876"
Sep 19 10:44:07.392193 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:44:07.392168    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-gk4r6\" is forbidden: User \"system:node:ip-10-0-142-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-165.ec2.internal' and this object" podUID="2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gk4r6"
Sep 19 10:44:07.396834 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:07.396806    2576 scope.go:117] "RemoveContainer" containerID="4866fe2ba15ac7de070e59c2599bea7bee2f6335bfbb3e65c012ba43322db876"
Sep 19 10:44:07.397088 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:44:07.397070    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4866fe2ba15ac7de070e59c2599bea7bee2f6335bfbb3e65c012ba43322db876\": container with ID starting with 4866fe2ba15ac7de070e59c2599bea7bee2f6335bfbb3e65c012ba43322db876 not found: ID does not exist" containerID="4866fe2ba15ac7de070e59c2599bea7bee2f6335bfbb3e65c012ba43322db876"
Sep 19 10:44:07.397160 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:07.397094    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4866fe2ba15ac7de070e59c2599bea7bee2f6335bfbb3e65c012ba43322db876"} err="failed to get container status \"4866fe2ba15ac7de070e59c2599bea7bee2f6335bfbb3e65c012ba43322db876\": rpc error: code = NotFound desc = could not find container \"4866fe2ba15ac7de070e59c2599bea7bee2f6335bfbb3e65c012ba43322db876\": container with ID starting with 4866fe2ba15ac7de070e59c2599bea7bee2f6335bfbb3e65c012ba43322db876 not found: ID does not exist"
Sep 19 10:44:17.390235 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:17.390205    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g"
Sep 19 10:44:29.996718 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:29.996686    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g"]
Sep 19 10:44:29.997123 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:29.996857    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g" podUID="068969ba-6b70-4cc0-a0bd-c0fc4601ee1f" containerName="manager" containerID="cri-o://17af5bfe115f5e4b295d623f8647f4d99f412ae8d246f5d1c6acbf9ead3e3ae8" gracePeriod=10
Sep 19 10:44:30.243003 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:30.242982    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g"
Sep 19 10:44:30.309057 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:30.308965    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nw27\" (UniqueName: \"kubernetes.io/projected/068969ba-6b70-4cc0-a0bd-c0fc4601ee1f-kube-api-access-8nw27\") pod \"068969ba-6b70-4cc0-a0bd-c0fc4601ee1f\" (UID: \"068969ba-6b70-4cc0-a0bd-c0fc4601ee1f\") "
Sep 19 10:44:30.309057 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:30.309032    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/068969ba-6b70-4cc0-a0bd-c0fc4601ee1f-extensions-socket-volume\") pod \"068969ba-6b70-4cc0-a0bd-c0fc4601ee1f\" (UID: \"068969ba-6b70-4cc0-a0bd-c0fc4601ee1f\") "
Sep 19 10:44:30.309500 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:30.309475    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/068969ba-6b70-4cc0-a0bd-c0fc4601ee1f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "068969ba-6b70-4cc0-a0bd-c0fc4601ee1f" (UID: "068969ba-6b70-4cc0-a0bd-c0fc4601ee1f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:44:30.310902 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:30.310869    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/068969ba-6b70-4cc0-a0bd-c0fc4601ee1f-kube-api-access-8nw27" (OuterVolumeSpecName: "kube-api-access-8nw27") pod "068969ba-6b70-4cc0-a0bd-c0fc4601ee1f" (UID: "068969ba-6b70-4cc0-a0bd-c0fc4601ee1f"). InnerVolumeSpecName "kube-api-access-8nw27". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:44:30.410483 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:30.410459    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nw27\" (UniqueName: \"kubernetes.io/projected/068969ba-6b70-4cc0-a0bd-c0fc4601ee1f-kube-api-access-8nw27\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:44:30.410483 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:30.410480    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/068969ba-6b70-4cc0-a0bd-c0fc4601ee1f-extensions-socket-volume\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:44:30.502030 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:30.502003    2576 generic.go:356] "Generic (PLEG): container finished" podID="068969ba-6b70-4cc0-a0bd-c0fc4601ee1f" containerID="17af5bfe115f5e4b295d623f8647f4d99f412ae8d246f5d1c6acbf9ead3e3ae8" exitCode=0
Sep 19 10:44:30.502146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:30.502057    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g" event={"ID":"068969ba-6b70-4cc0-a0bd-c0fc4601ee1f","Type":"ContainerDied","Data":"17af5bfe115f5e4b295d623f8647f4d99f412ae8d246f5d1c6acbf9ead3e3ae8"}
Sep 19 10:44:30.502146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:30.502096    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g" event={"ID":"068969ba-6b70-4cc0-a0bd-c0fc4601ee1f","Type":"ContainerDied","Data":"0494bb8515ffa44493e81a9034b3672b99c904ecad4695117bd27eeac8fd2c9e"}
Sep 19 10:44:30.502146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:30.502117    2576 scope.go:117] "RemoveContainer" containerID="17af5bfe115f5e4b295d623f8647f4d99f412ae8d246f5d1c6acbf9ead3e3ae8"
Sep 19 10:44:30.502146 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:30.502058    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g"
Sep 19 10:44:30.520381 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:30.520362    2576 scope.go:117] "RemoveContainer" containerID="17af5bfe115f5e4b295d623f8647f4d99f412ae8d246f5d1c6acbf9ead3e3ae8"
Sep 19 10:44:30.520638 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:44:30.520619    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17af5bfe115f5e4b295d623f8647f4d99f412ae8d246f5d1c6acbf9ead3e3ae8\": container with ID starting with 17af5bfe115f5e4b295d623f8647f4d99f412ae8d246f5d1c6acbf9ead3e3ae8 not found: ID does not exist" containerID="17af5bfe115f5e4b295d623f8647f4d99f412ae8d246f5d1c6acbf9ead3e3ae8"
Sep 19 10:44:30.520744 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:30.520645    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17af5bfe115f5e4b295d623f8647f4d99f412ae8d246f5d1c6acbf9ead3e3ae8"} err="failed to get container status \"17af5bfe115f5e4b295d623f8647f4d99f412ae8d246f5d1c6acbf9ead3e3ae8\": rpc error: code = NotFound desc = could not find container \"17af5bfe115f5e4b295d623f8647f4d99f412ae8d246f5d1c6acbf9ead3e3ae8\": container with ID starting with 17af5bfe115f5e4b295d623f8647f4d99f412ae8d246f5d1c6acbf9ead3e3ae8 not found: ID does not exist"
Sep 19 10:44:30.525688 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:30.525666    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g"]
Sep 19 10:44:30.529412 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:30.529392    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-mgh2g"]
Sep 19 10:44:31.000183 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:31.000157    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="068969ba-6b70-4cc0-a0bd-c0fc4601ee1f" path="/var/lib/kubelet/pods/068969ba-6b70-4cc0-a0bd-c0fc4601ee1f/volumes"
Sep 19 10:44:50.186510 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.186476    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-9xgzr"]
Sep 19 10:44:50.186995 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.186826    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee" containerName="manager"
Sep 19 10:44:50.186995 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.186838    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee" containerName="manager"
Sep 19 10:44:50.186995 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.186856    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="068969ba-6b70-4cc0-a0bd-c0fc4601ee1f" containerName="manager"
Sep 19 10:44:50.186995 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.186862    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="068969ba-6b70-4cc0-a0bd-c0fc4601ee1f" containerName="manager"
Sep 19 10:44:50.186995 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.186923    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bbe01ff-81c4-4a5b-b2c1-aa05316d12ee" containerName="manager"
Sep 19 10:44:50.186995 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.186931    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="068969ba-6b70-4cc0-a0bd-c0fc4601ee1f" containerName="manager"
Sep 19 10:44:50.190865 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.190849    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-9xgzr"
Sep 19 10:44:50.193230 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.193204    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 19 10:44:50.193348 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.193246    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ss7b2\""
Sep 19 10:44:50.194248 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.194226    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 19 10:44:50.194456 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.194257    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 19 10:44:50.196915 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.196896    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-9xgzr"]
Sep 19 10:44:50.259536 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.259510    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d326e4ba-5dd8-4269-bc1e-d8465fe05908-config-file\") pod \"limitador-limitador-8588749977-9xgzr\" (UID: \"d326e4ba-5dd8-4269-bc1e-d8465fe05908\") " pod="kuadrant-system/limitador-limitador-8588749977-9xgzr"
Sep 19 10:44:50.259703 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.259562    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcrmf\" (UniqueName: \"kubernetes.io/projected/d326e4ba-5dd8-4269-bc1e-d8465fe05908-kube-api-access-rcrmf\") pod \"limitador-limitador-8588749977-9xgzr\" (UID: \"d326e4ba-5dd8-4269-bc1e-d8465fe05908\") " pod="kuadrant-system/limitador-limitador-8588749977-9xgzr"
Sep 19 10:44:50.286734 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.286707    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-9xgzr"]
Sep 19 10:44:50.360562 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.360528    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d326e4ba-5dd8-4269-bc1e-d8465fe05908-config-file\") pod \"limitador-limitador-8588749977-9xgzr\" (UID: \"d326e4ba-5dd8-4269-bc1e-d8465fe05908\") " pod="kuadrant-system/limitador-limitador-8588749977-9xgzr"
Sep 19 10:44:50.360734 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.360594    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcrmf\" (UniqueName: \"kubernetes.io/projected/d326e4ba-5dd8-4269-bc1e-d8465fe05908-kube-api-access-rcrmf\") pod \"limitador-limitador-8588749977-9xgzr\" (UID: \"d326e4ba-5dd8-4269-bc1e-d8465fe05908\") " pod="kuadrant-system/limitador-limitador-8588749977-9xgzr"
Sep 19 10:44:50.361133 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.361116    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d326e4ba-5dd8-4269-bc1e-d8465fe05908-config-file\") pod \"limitador-limitador-8588749977-9xgzr\" (UID: \"d326e4ba-5dd8-4269-bc1e-d8465fe05908\") " pod="kuadrant-system/limitador-limitador-8588749977-9xgzr"
Sep 19 10:44:50.368321 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.368302    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcrmf\" (UniqueName: \"kubernetes.io/projected/d326e4ba-5dd8-4269-bc1e-d8465fe05908-kube-api-access-rcrmf\") pod \"limitador-limitador-8588749977-9xgzr\" (UID: \"d326e4ba-5dd8-4269-bc1e-d8465fe05908\") " pod="kuadrant-system/limitador-limitador-8588749977-9xgzr"
Sep 19 10:44:50.502169 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.502088    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-9xgzr"
Sep 19 10:44:50.770763 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.770739    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-9xgzr"]
Sep 19 10:44:50.772773 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:44:50.772739    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd326e4ba_5dd8_4269_bc1e_d8465fe05908.slice/crio-e361d54bd8906bba4ada9895979fb7743a9245b3170cba585da0045daad549dd WatchSource:0}: Error finding container e361d54bd8906bba4ada9895979fb7743a9245b3170cba585da0045daad549dd: Status 404 returned error can't find the container with id e361d54bd8906bba4ada9895979fb7743a9245b3170cba585da0045daad549dd
Sep 19 10:44:50.984618 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.984592    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-n4r8x"]
Sep 19 10:44:50.989262 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.989244    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-n4r8x"
Sep 19 10:44:50.991604 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:50.991587    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-g6vhl\""
Sep 19 10:44:51.002470 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:51.002450    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-n4r8x"]
Sep 19 10:44:51.065666 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:51.065599    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sspf7\" (UniqueName: \"kubernetes.io/projected/bfa60ef1-accb-4769-b327-76b8f95c0fdd-kube-api-access-sspf7\") pod \"authorino-7bcfd8f9cd-n4r8x\" (UID: \"bfa60ef1-accb-4769-b327-76b8f95c0fdd\") " pod="kuadrant-system/authorino-7bcfd8f9cd-n4r8x"
Sep 19 10:44:51.166085 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:51.166059    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sspf7\" (UniqueName: \"kubernetes.io/projected/bfa60ef1-accb-4769-b327-76b8f95c0fdd-kube-api-access-sspf7\") pod \"authorino-7bcfd8f9cd-n4r8x\" (UID: \"bfa60ef1-accb-4769-b327-76b8f95c0fdd\") " pod="kuadrant-system/authorino-7bcfd8f9cd-n4r8x"
Sep 19 10:44:51.173504 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:51.173486    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sspf7\" (UniqueName: \"kubernetes.io/projected/bfa60ef1-accb-4769-b327-76b8f95c0fdd-kube-api-access-sspf7\") pod \"authorino-7bcfd8f9cd-n4r8x\" (UID: \"bfa60ef1-accb-4769-b327-76b8f95c0fdd\") " pod="kuadrant-system/authorino-7bcfd8f9cd-n4r8x"
Sep 19 10:44:51.307131 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:51.307108    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-n4r8x"
Sep 19 10:44:51.307824 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:51.307807    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-lnmq9"]
Sep 19 10:44:51.310827 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:51.310814    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-lnmq9"
Sep 19 10:44:51.320224 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:51.320202    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-lnmq9"]
Sep 19 10:44:51.368621 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:51.368588    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsggf\" (UniqueName: \"kubernetes.io/projected/445167ab-d8fd-4400-aa48-a9223c761008-kube-api-access-xsggf\") pod \"authorino-59c974bd48-lnmq9\" (UID: \"445167ab-d8fd-4400-aa48-a9223c761008\") " pod="kuadrant-system/authorino-59c974bd48-lnmq9"
Sep 19 10:44:51.469937 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:51.469911    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsggf\" (UniqueName: \"kubernetes.io/projected/445167ab-d8fd-4400-aa48-a9223c761008-kube-api-access-xsggf\") pod \"authorino-59c974bd48-lnmq9\" (UID: \"445167ab-d8fd-4400-aa48-a9223c761008\") " pod="kuadrant-system/authorino-59c974bd48-lnmq9"
Sep 19 10:44:51.477651 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:51.477627    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsggf\" (UniqueName: \"kubernetes.io/projected/445167ab-d8fd-4400-aa48-a9223c761008-kube-api-access-xsggf\") pod \"authorino-59c974bd48-lnmq9\" (UID: \"445167ab-d8fd-4400-aa48-a9223c761008\") " pod="kuadrant-system/authorino-59c974bd48-lnmq9"
Sep 19 10:44:51.577247 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:51.577191    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-n4r8x"]
Sep 19 10:44:51.579997 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:44:51.579972    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbfa60ef1_accb_4769_b327_76b8f95c0fdd.slice/crio-6bf0689b1923006ab60bb7887135124258ed90e248e7f377a2aac1d50a43ae81 WatchSource:0}: Error finding container 6bf0689b1923006ab60bb7887135124258ed90e248e7f377a2aac1d50a43ae81: Status 404 returned error can't find the container with id 6bf0689b1923006ab60bb7887135124258ed90e248e7f377a2aac1d50a43ae81
Sep 19 10:44:51.605621 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:51.605590    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-9xgzr" event={"ID":"d326e4ba-5dd8-4269-bc1e-d8465fe05908","Type":"ContainerStarted","Data":"e361d54bd8906bba4ada9895979fb7743a9245b3170cba585da0045daad549dd"}
Sep 19 10:44:51.606738 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:51.606717    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-n4r8x" event={"ID":"bfa60ef1-accb-4769-b327-76b8f95c0fdd","Type":"ContainerStarted","Data":"6bf0689b1923006ab60bb7887135124258ed90e248e7f377a2aac1d50a43ae81"}
Sep 19 10:44:51.623941 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:51.623915    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-lnmq9"
Sep 19 10:44:51.902763 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:51.902734    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-lnmq9"]
Sep 19 10:44:51.940318 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:44:51.940285    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod445167ab_d8fd_4400_aa48_a9223c761008.slice/crio-6303dcf2b6e4d95cbe997d88541ccd0c84f63b6d16fd6a5aee3ce91335a45483 WatchSource:0}: Error finding container 6303dcf2b6e4d95cbe997d88541ccd0c84f63b6d16fd6a5aee3ce91335a45483: Status 404 returned error can't find the container with id 6303dcf2b6e4d95cbe997d88541ccd0c84f63b6d16fd6a5aee3ce91335a45483
Sep 19 10:44:52.613824 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:52.613773    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-lnmq9" event={"ID":"445167ab-d8fd-4400-aa48-a9223c761008","Type":"ContainerStarted","Data":"6303dcf2b6e4d95cbe997d88541ccd0c84f63b6d16fd6a5aee3ce91335a45483"}
Sep 19 10:44:52.616023 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:52.615994    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-9xgzr" event={"ID":"d326e4ba-5dd8-4269-bc1e-d8465fe05908","Type":"ContainerStarted","Data":"30a40ffde38dbbd79df4d24b0473ae163773afac73e1d8d36a661ba8bcc09f01"}
Sep 19 10:44:52.616230 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:52.616207    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-9xgzr"
Sep 19 10:44:52.632223 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:52.632164    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-9xgzr" podStartSLOduration=1.412257071 podStartE2EDuration="2.632145227s" totalImagesPullingTime="1.219888156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:44:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:44:52.630813311 +0000 UTC m=+502.127119404" watchObservedRunningTime="2026-09-19 10:44:52.632145227 +0000 UTC m=+502.128451320"
Sep 19 10:44:53.622868 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:53.622835    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-n4r8x" event={"ID":"bfa60ef1-accb-4769-b327-76b8f95c0fdd","Type":"ContainerStarted","Data":"61656830fc6e2326a22a956cc8b179819d9d1b6f49cacf25297d639427fc7e66"}
Sep 19 10:44:53.624470 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:53.624437    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-lnmq9" event={"ID":"445167ab-d8fd-4400-aa48-a9223c761008","Type":"ContainerStarted","Data":"1bf57ed0ecea0696d0130d6f9b9548b3a435dfa622476c56723bc02c712b0da5"}
Sep 19 10:44:53.637107 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:53.637012    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-n4r8x" podStartSLOduration=1.828144139 podStartE2EDuration="3.636999555s" totalImagesPullingTime="1.808855416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:44:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:44:53.635901967 +0000 UTC m=+503.132208084" watchObservedRunningTime="2026-09-19 10:44:53.636999555 +0000 UTC m=+503.133305646"
Sep 19 10:44:53.650289 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:53.650238    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-lnmq9" podStartSLOduration=1.1969981 podStartE2EDuration="2.650219782s" totalImagesPullingTime="1.453221682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:44:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:44:53.64894512 +0000 UTC m=+503.145251224" watchObservedRunningTime="2026-09-19 10:44:53.650219782 +0000 UTC m=+503.146525875"
Sep 19 10:44:53.673535 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:53.673511    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-n4r8x"]
Sep 19 10:44:55.633606 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:55.633551    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-n4r8x" podUID="bfa60ef1-accb-4769-b327-76b8f95c0fdd" containerName="authorino" containerID="cri-o://61656830fc6e2326a22a956cc8b179819d9d1b6f49cacf25297d639427fc7e66" gracePeriod=30
Sep 19 10:44:55.879181 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:55.879155    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-n4r8x"
Sep 19 10:44:56.009191 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:56.009130    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sspf7\" (UniqueName: \"kubernetes.io/projected/bfa60ef1-accb-4769-b327-76b8f95c0fdd-kube-api-access-sspf7\") pod \"bfa60ef1-accb-4769-b327-76b8f95c0fdd\" (UID: \"bfa60ef1-accb-4769-b327-76b8f95c0fdd\") "
Sep 19 10:44:56.011319 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:56.011297    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfa60ef1-accb-4769-b327-76b8f95c0fdd-kube-api-access-sspf7" (OuterVolumeSpecName: "kube-api-access-sspf7") pod "bfa60ef1-accb-4769-b327-76b8f95c0fdd" (UID: "bfa60ef1-accb-4769-b327-76b8f95c0fdd"). InnerVolumeSpecName "kube-api-access-sspf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:44:56.111659 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:56.111636    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sspf7\" (UniqueName: \"kubernetes.io/projected/bfa60ef1-accb-4769-b327-76b8f95c0fdd-kube-api-access-sspf7\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:44:56.639059 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:56.639010    2576 generic.go:356] "Generic (PLEG): container finished" podID="bfa60ef1-accb-4769-b327-76b8f95c0fdd" containerID="61656830fc6e2326a22a956cc8b179819d9d1b6f49cacf25297d639427fc7e66" exitCode=0
Sep 19 10:44:56.639520 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:56.639072    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-n4r8x"
Sep 19 10:44:56.639520 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:56.639091    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-n4r8x" event={"ID":"bfa60ef1-accb-4769-b327-76b8f95c0fdd","Type":"ContainerDied","Data":"61656830fc6e2326a22a956cc8b179819d9d1b6f49cacf25297d639427fc7e66"}
Sep 19 10:44:56.639520 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:56.639131    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-n4r8x" event={"ID":"bfa60ef1-accb-4769-b327-76b8f95c0fdd","Type":"ContainerDied","Data":"6bf0689b1923006ab60bb7887135124258ed90e248e7f377a2aac1d50a43ae81"}
Sep 19 10:44:56.639520 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:56.639148    2576 scope.go:117] "RemoveContainer" containerID="61656830fc6e2326a22a956cc8b179819d9d1b6f49cacf25297d639427fc7e66"
Sep 19 10:44:56.647945 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:56.647930    2576 scope.go:117] "RemoveContainer" containerID="61656830fc6e2326a22a956cc8b179819d9d1b6f49cacf25297d639427fc7e66"
Sep 19 10:44:56.648256 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:44:56.648239    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61656830fc6e2326a22a956cc8b179819d9d1b6f49cacf25297d639427fc7e66\": container with ID starting with 61656830fc6e2326a22a956cc8b179819d9d1b6f49cacf25297d639427fc7e66 not found: ID does not exist" containerID="61656830fc6e2326a22a956cc8b179819d9d1b6f49cacf25297d639427fc7e66"
Sep 19 10:44:56.648318 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:56.648264    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61656830fc6e2326a22a956cc8b179819d9d1b6f49cacf25297d639427fc7e66"} err="failed to get container status \"61656830fc6e2326a22a956cc8b179819d9d1b6f49cacf25297d639427fc7e66\": rpc error: code = NotFound desc = could not find container \"61656830fc6e2326a22a956cc8b179819d9d1b6f49cacf25297d639427fc7e66\": container with ID starting with 61656830fc6e2326a22a956cc8b179819d9d1b6f49cacf25297d639427fc7e66 not found: ID does not exist"
Sep 19 10:44:56.660123 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:56.660105    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-n4r8x"]
Sep 19 10:44:56.663594 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:56.663576    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-n4r8x"]
Sep 19 10:44:57.001369 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:44:57.001283    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfa60ef1-accb-4769-b327-76b8f95c0fdd" path="/var/lib/kubelet/pods/bfa60ef1-accb-4769-b327-76b8f95c0fdd/volumes"
Sep 19 10:45:03.139546 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:03.139507    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-9xgzr"]
Sep 19 10:45:03.140127 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:03.139875    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-9xgzr" podUID="d326e4ba-5dd8-4269-bc1e-d8465fe05908" containerName="limitador" containerID="cri-o://30a40ffde38dbbd79df4d24b0473ae163773afac73e1d8d36a661ba8bcc09f01" gracePeriod=30
Sep 19 10:45:03.143946 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:03.143919    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-9xgzr"
Sep 19 10:45:03.672418 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:03.672392    2576 generic.go:356] "Generic (PLEG): container finished" podID="d326e4ba-5dd8-4269-bc1e-d8465fe05908" containerID="30a40ffde38dbbd79df4d24b0473ae163773afac73e1d8d36a661ba8bcc09f01" exitCode=0
Sep 19 10:45:03.672559 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:03.672466    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-9xgzr" event={"ID":"d326e4ba-5dd8-4269-bc1e-d8465fe05908","Type":"ContainerDied","Data":"30a40ffde38dbbd79df4d24b0473ae163773afac73e1d8d36a661ba8bcc09f01"}
Sep 19 10:45:03.687745 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:03.687727    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-9xgzr"
Sep 19 10:45:03.768556 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:03.768492    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d326e4ba-5dd8-4269-bc1e-d8465fe05908-config-file\") pod \"d326e4ba-5dd8-4269-bc1e-d8465fe05908\" (UID: \"d326e4ba-5dd8-4269-bc1e-d8465fe05908\") "
Sep 19 10:45:03.768664 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:03.768633    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcrmf\" (UniqueName: \"kubernetes.io/projected/d326e4ba-5dd8-4269-bc1e-d8465fe05908-kube-api-access-rcrmf\") pod \"d326e4ba-5dd8-4269-bc1e-d8465fe05908\" (UID: \"d326e4ba-5dd8-4269-bc1e-d8465fe05908\") "
Sep 19 10:45:03.768831 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:03.768812    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d326e4ba-5dd8-4269-bc1e-d8465fe05908-config-file" (OuterVolumeSpecName: "config-file") pod "d326e4ba-5dd8-4269-bc1e-d8465fe05908" (UID: "d326e4ba-5dd8-4269-bc1e-d8465fe05908"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:45:03.770735 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:03.770712    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d326e4ba-5dd8-4269-bc1e-d8465fe05908-kube-api-access-rcrmf" (OuterVolumeSpecName: "kube-api-access-rcrmf") pod "d326e4ba-5dd8-4269-bc1e-d8465fe05908" (UID: "d326e4ba-5dd8-4269-bc1e-d8465fe05908"). InnerVolumeSpecName "kube-api-access-rcrmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:45:03.870151 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:03.870125    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcrmf\" (UniqueName: \"kubernetes.io/projected/d326e4ba-5dd8-4269-bc1e-d8465fe05908-kube-api-access-rcrmf\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:45:03.870151 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:03.870147    2576 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d326e4ba-5dd8-4269-bc1e-d8465fe05908-config-file\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:45:04.679059 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:04.679023    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-9xgzr"
Sep 19 10:45:04.679460 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:04.679077    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-9xgzr" event={"ID":"d326e4ba-5dd8-4269-bc1e-d8465fe05908","Type":"ContainerDied","Data":"e361d54bd8906bba4ada9895979fb7743a9245b3170cba585da0045daad549dd"}
Sep 19 10:45:04.679460 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:04.679137    2576 scope.go:117] "RemoveContainer" containerID="30a40ffde38dbbd79df4d24b0473ae163773afac73e1d8d36a661ba8bcc09f01"
Sep 19 10:45:04.698678 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:04.698644    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-9xgzr"]
Sep 19 10:45:04.700865 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:04.700845    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-9xgzr"]
Sep 19 10:45:05.000998 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:05.000933    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d326e4ba-5dd8-4269-bc1e-d8465fe05908" path="/var/lib/kubelet/pods/d326e4ba-5dd8-4269-bc1e-d8465fe05908/volumes"
Sep 19 10:45:07.373844 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.373809    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-x4lxv"]
Sep 19 10:45:07.374360 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.374341    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfa60ef1-accb-4769-b327-76b8f95c0fdd" containerName="authorino"
Sep 19 10:45:07.374443 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.374363    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfa60ef1-accb-4769-b327-76b8f95c0fdd" containerName="authorino"
Sep 19 10:45:07.374443 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.374407    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d326e4ba-5dd8-4269-bc1e-d8465fe05908" containerName="limitador"
Sep 19 10:45:07.374443 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.374416    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d326e4ba-5dd8-4269-bc1e-d8465fe05908" containerName="limitador"
Sep 19 10:45:07.374591 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.374504    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d326e4ba-5dd8-4269-bc1e-d8465fe05908" containerName="limitador"
Sep 19 10:45:07.374591 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.374522    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfa60ef1-accb-4769-b327-76b8f95c0fdd" containerName="authorino"
Sep 19 10:45:07.378804 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.378779    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x4lxv"
Sep 19 10:45:07.381492 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.381463    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 10:45:07.381492 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.381480    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-v8927\""
Sep 19 10:45:07.381680 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.381463    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 19 10:45:07.381680 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.381547    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 19 10:45:07.390977 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.390955    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-x4lxv"]
Sep 19 10:45:07.497369 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.497338    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf9nh\" (UniqueName: \"kubernetes.io/projected/758ba15e-8cf6-41d9-9f30-579dc0ed6146-kube-api-access-gf9nh\") pod \"postgres-69fb4594cf-x4lxv\" (UID: \"758ba15e-8cf6-41d9-9f30-579dc0ed6146\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x4lxv"
Sep 19 10:45:07.497521 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.497391    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/758ba15e-8cf6-41d9-9f30-579dc0ed6146-data\") pod \"postgres-69fb4594cf-x4lxv\" (UID: \"758ba15e-8cf6-41d9-9f30-579dc0ed6146\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x4lxv"
Sep 19 10:45:07.598241 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.598213    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf9nh\" (UniqueName: \"kubernetes.io/projected/758ba15e-8cf6-41d9-9f30-579dc0ed6146-kube-api-access-gf9nh\") pod \"postgres-69fb4594cf-x4lxv\" (UID: \"758ba15e-8cf6-41d9-9f30-579dc0ed6146\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x4lxv"
Sep 19 10:45:07.598398 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.598264    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/758ba15e-8cf6-41d9-9f30-579dc0ed6146-data\") pod \"postgres-69fb4594cf-x4lxv\" (UID: \"758ba15e-8cf6-41d9-9f30-579dc0ed6146\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x4lxv"
Sep 19 10:45:07.598623 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.598604    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/758ba15e-8cf6-41d9-9f30-579dc0ed6146-data\") pod \"postgres-69fb4594cf-x4lxv\" (UID: \"758ba15e-8cf6-41d9-9f30-579dc0ed6146\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x4lxv"
Sep 19 10:45:07.605831 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.605803    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf9nh\" (UniqueName: \"kubernetes.io/projected/758ba15e-8cf6-41d9-9f30-579dc0ed6146-kube-api-access-gf9nh\") pod \"postgres-69fb4594cf-x4lxv\" (UID: \"758ba15e-8cf6-41d9-9f30-579dc0ed6146\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x4lxv"
Sep 19 10:45:07.689874 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.689811    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x4lxv"
Sep 19 10:45:07.956866 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:07.956839    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-x4lxv"]
Sep 19 10:45:07.958721 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:45:07.958690    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod758ba15e_8cf6_41d9_9f30_579dc0ed6146.slice/crio-f6ce7a4815723df2230ab611aea7fd7e32d4ca37099e9e51e83be5b9d19c1291 WatchSource:0}: Error finding container f6ce7a4815723df2230ab611aea7fd7e32d4ca37099e9e51e83be5b9d19c1291: Status 404 returned error can't find the container with id f6ce7a4815723df2230ab611aea7fd7e32d4ca37099e9e51e83be5b9d19c1291
Sep 19 10:45:08.702167 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:08.702125    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x4lxv" event={"ID":"758ba15e-8cf6-41d9-9f30-579dc0ed6146","Type":"ContainerStarted","Data":"f6ce7a4815723df2230ab611aea7fd7e32d4ca37099e9e51e83be5b9d19c1291"}
Sep 19 10:45:13.738593 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:13.738552    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x4lxv" event={"ID":"758ba15e-8cf6-41d9-9f30-579dc0ed6146","Type":"ContainerStarted","Data":"53b26cc15595d9ffa6050c33a9f9bc1f7eccc8453fab8634f89ef63c4f18918f"}
Sep 19 10:45:13.739128 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:13.738699    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x4lxv"
Sep 19 10:45:13.754300 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:13.754261    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x4lxv" podStartSLOduration=1.823869498 podStartE2EDuration="6.754247778s" totalImagesPullingTime="4.93037828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:45:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:45:13.75245896 +0000 UTC m=+523.248765053" watchObservedRunningTime="2026-09-19 10:45:13.754247778 +0000 UTC m=+523.250553871"
Sep 19 10:45:19.773881 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:19.773852    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x4lxv"
Sep 19 10:45:20.291252 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:20.291219    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-5lvbt"]
Sep 19 10:45:20.294522 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:20.294502    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-5lvbt"
Sep 19 10:45:20.300859 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:20.300833    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-5lvbt"]
Sep 19 10:45:20.409158 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:20.409129    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5wvh\" (UniqueName: \"kubernetes.io/projected/de83f3a7-d894-41be-aa11-911ed1017945-kube-api-access-n5wvh\") pod \"authorino-557f876f69-5lvbt\" (UID: \"de83f3a7-d894-41be-aa11-911ed1017945\") " pod="kuadrant-system/authorino-557f876f69-5lvbt"
Sep 19 10:45:20.510410 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:20.510380    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5wvh\" (UniqueName: \"kubernetes.io/projected/de83f3a7-d894-41be-aa11-911ed1017945-kube-api-access-n5wvh\") pod \"authorino-557f876f69-5lvbt\" (UID: \"de83f3a7-d894-41be-aa11-911ed1017945\") " pod="kuadrant-system/authorino-557f876f69-5lvbt"
Sep 19 10:45:20.518798 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:20.518761    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5wvh\" (UniqueName: \"kubernetes.io/projected/de83f3a7-d894-41be-aa11-911ed1017945-kube-api-access-n5wvh\") pod \"authorino-557f876f69-5lvbt\" (UID: \"de83f3a7-d894-41be-aa11-911ed1017945\") " pod="kuadrant-system/authorino-557f876f69-5lvbt"
Sep 19 10:45:20.522469 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:20.522441    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-5lvbt"]
Sep 19 10:45:20.522676 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:20.522662    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-5lvbt"
Sep 19 10:45:20.547258 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:20.547196    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-799ff98669-fqhdr"]
Sep 19 10:45:20.550435 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:20.550415    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-799ff98669-fqhdr"
Sep 19 10:45:20.554892 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:20.554869    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-799ff98669-fqhdr"]
Sep 19 10:45:20.712263 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:20.712228    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-868cw\" (UniqueName: \"kubernetes.io/projected/63256afe-3f42-4a24-957a-e8a478d7f44e-kube-api-access-868cw\") pod \"authorino-799ff98669-fqhdr\" (UID: \"63256afe-3f42-4a24-957a-e8a478d7f44e\") " pod="kuadrant-system/authorino-799ff98669-fqhdr"
Sep 19 10:45:20.797916 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:20.797833    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-5lvbt"]
Sep 19 10:45:20.800298 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:45:20.800261    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde83f3a7_d894_41be_aa11_911ed1017945.slice/crio-d02e2399221440be7e8c65eacbe9b9685f6e77039d9f572c147b92e16b8f8b0d WatchSource:0}: Error finding container d02e2399221440be7e8c65eacbe9b9685f6e77039d9f572c147b92e16b8f8b0d: Status 404 returned error can't find the container with id d02e2399221440be7e8c65eacbe9b9685f6e77039d9f572c147b92e16b8f8b0d
Sep 19 10:45:20.812724 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:20.812704    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-868cw\" (UniqueName: \"kubernetes.io/projected/63256afe-3f42-4a24-957a-e8a478d7f44e-kube-api-access-868cw\") pod \"authorino-799ff98669-fqhdr\" (UID: \"63256afe-3f42-4a24-957a-e8a478d7f44e\") " pod="kuadrant-system/authorino-799ff98669-fqhdr"
Sep 19 10:45:20.820346 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:20.820320    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-868cw\" (UniqueName: \"kubernetes.io/projected/63256afe-3f42-4a24-957a-e8a478d7f44e-kube-api-access-868cw\") pod \"authorino-799ff98669-fqhdr\" (UID: \"63256afe-3f42-4a24-957a-e8a478d7f44e\") " pod="kuadrant-system/authorino-799ff98669-fqhdr"
Sep 19 10:45:20.861111 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:20.861090    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-799ff98669-fqhdr"
Sep 19 10:45:21.126950 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:21.126926    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-799ff98669-fqhdr"]
Sep 19 10:45:21.128933 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:45:21.128906    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63256afe_3f42_4a24_957a_e8a478d7f44e.slice/crio-c6ba49add8a2a74f4753f32312a93b8e5389f3726223bdb1f57022bafac5a1b8 WatchSource:0}: Error finding container c6ba49add8a2a74f4753f32312a93b8e5389f3726223bdb1f57022bafac5a1b8: Status 404 returned error can't find the container with id c6ba49add8a2a74f4753f32312a93b8e5389f3726223bdb1f57022bafac5a1b8
Sep 19 10:45:21.780687 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:21.780605    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-5lvbt" event={"ID":"de83f3a7-d894-41be-aa11-911ed1017945","Type":"ContainerStarted","Data":"efb99787fc2888d48ec501e7485f6aef7b4f2207b1ccfc4cd88fb81e99f28d84"}
Sep 19 10:45:21.780687 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:21.780649    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-5lvbt" event={"ID":"de83f3a7-d894-41be-aa11-911ed1017945","Type":"ContainerStarted","Data":"d02e2399221440be7e8c65eacbe9b9685f6e77039d9f572c147b92e16b8f8b0d"}
Sep 19 10:45:21.780687 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:21.780635    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-557f876f69-5lvbt" podUID="de83f3a7-d894-41be-aa11-911ed1017945" containerName="authorino" containerID="cri-o://efb99787fc2888d48ec501e7485f6aef7b4f2207b1ccfc4cd88fb81e99f28d84" gracePeriod=30
Sep 19 10:45:21.782444 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:21.782419    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-799ff98669-fqhdr" event={"ID":"63256afe-3f42-4a24-957a-e8a478d7f44e","Type":"ContainerStarted","Data":"ddac78d74830d437944b87fdf89ea6ea6fd26e88ce1a9e3816cacda31b1ffa0f"}
Sep 19 10:45:21.782572 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:21.782452    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-799ff98669-fqhdr" event={"ID":"63256afe-3f42-4a24-957a-e8a478d7f44e","Type":"ContainerStarted","Data":"c6ba49add8a2a74f4753f32312a93b8e5389f3726223bdb1f57022bafac5a1b8"}
Sep 19 10:45:21.795420 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:21.795378    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-557f876f69-5lvbt" podStartSLOduration=1.340591268 podStartE2EDuration="1.795364437s" totalImagesPullingTime="454.773169ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:45:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:45:21.794446235 +0000 UTC m=+531.290752328" watchObservedRunningTime="2026-09-19 10:45:21.795364437 +0000 UTC m=+531.291670526"
Sep 19 10:45:21.829989 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:21.829947    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-799ff98669-fqhdr" podStartSLOduration=1.3733051600000001 podStartE2EDuration="1.829929914s" totalImagesPullingTime="456.624754ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:45:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:45:21.807312184 +0000 UTC m=+531.303618274" watchObservedRunningTime="2026-09-19 10:45:21.829929914 +0000 UTC m=+531.326236005"
Sep 19 10:45:21.830467 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:21.830450    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-lnmq9"]
Sep 19 10:45:21.830625 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:21.830605    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-lnmq9" podUID="445167ab-d8fd-4400-aa48-a9223c761008" containerName="authorino" containerID="cri-o://1bf57ed0ecea0696d0130d6f9b9548b3a435dfa622476c56723bc02c712b0da5" gracePeriod=30
Sep 19 10:45:21.903943 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:21.903918    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-659996695f-9jmnm"]
Sep 19 10:45:21.909496 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:21.909480    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-659996695f-9jmnm"
Sep 19 10:45:21.911847 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:21.911823    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 19 10:45:21.913285 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:21.913266    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-659996695f-9jmnm"]
Sep 19 10:45:22.023369 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.023338    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t42d\" (UniqueName: \"kubernetes.io/projected/97d6a09c-8ef5-40f9-841f-a8fee9914bca-kube-api-access-4t42d\") pod \"authorino-659996695f-9jmnm\" (UID: \"97d6a09c-8ef5-40f9-841f-a8fee9914bca\") " pod="kuadrant-system/authorino-659996695f-9jmnm"
Sep 19 10:45:22.023522 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.023378    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97d6a09c-8ef5-40f9-841f-a8fee9914bca-tls-cert\") pod \"authorino-659996695f-9jmnm\" (UID: \"97d6a09c-8ef5-40f9-841f-a8fee9914bca\") " pod="kuadrant-system/authorino-659996695f-9jmnm"
Sep 19 10:45:22.068473 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.068445    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-5lvbt"
Sep 19 10:45:22.088102 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.088077    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-lnmq9"
Sep 19 10:45:22.125001 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.124940    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t42d\" (UniqueName: \"kubernetes.io/projected/97d6a09c-8ef5-40f9-841f-a8fee9914bca-kube-api-access-4t42d\") pod \"authorino-659996695f-9jmnm\" (UID: \"97d6a09c-8ef5-40f9-841f-a8fee9914bca\") " pod="kuadrant-system/authorino-659996695f-9jmnm"
Sep 19 10:45:22.125001 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.124977    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97d6a09c-8ef5-40f9-841f-a8fee9914bca-tls-cert\") pod \"authorino-659996695f-9jmnm\" (UID: \"97d6a09c-8ef5-40f9-841f-a8fee9914bca\") " pod="kuadrant-system/authorino-659996695f-9jmnm"
Sep 19 10:45:22.127572 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.127552    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97d6a09c-8ef5-40f9-841f-a8fee9914bca-tls-cert\") pod \"authorino-659996695f-9jmnm\" (UID: \"97d6a09c-8ef5-40f9-841f-a8fee9914bca\") " pod="kuadrant-system/authorino-659996695f-9jmnm"
Sep 19 10:45:22.132589 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.132570    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t42d\" (UniqueName: \"kubernetes.io/projected/97d6a09c-8ef5-40f9-841f-a8fee9914bca-kube-api-access-4t42d\") pod \"authorino-659996695f-9jmnm\" (UID: \"97d6a09c-8ef5-40f9-841f-a8fee9914bca\") " pod="kuadrant-system/authorino-659996695f-9jmnm"
Sep 19 10:45:22.224719 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.224698    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-659996695f-9jmnm"
Sep 19 10:45:22.225938 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.225919    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5wvh\" (UniqueName: \"kubernetes.io/projected/de83f3a7-d894-41be-aa11-911ed1017945-kube-api-access-n5wvh\") pod \"de83f3a7-d894-41be-aa11-911ed1017945\" (UID: \"de83f3a7-d894-41be-aa11-911ed1017945\") "
Sep 19 10:45:22.225981 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.225973    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsggf\" (UniqueName: \"kubernetes.io/projected/445167ab-d8fd-4400-aa48-a9223c761008-kube-api-access-xsggf\") pod \"445167ab-d8fd-4400-aa48-a9223c761008\" (UID: \"445167ab-d8fd-4400-aa48-a9223c761008\") "
Sep 19 10:45:22.227891 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.227869    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de83f3a7-d894-41be-aa11-911ed1017945-kube-api-access-n5wvh" (OuterVolumeSpecName: "kube-api-access-n5wvh") pod "de83f3a7-d894-41be-aa11-911ed1017945" (UID: "de83f3a7-d894-41be-aa11-911ed1017945"). InnerVolumeSpecName "kube-api-access-n5wvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:45:22.227984 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.227939    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/445167ab-d8fd-4400-aa48-a9223c761008-kube-api-access-xsggf" (OuterVolumeSpecName: "kube-api-access-xsggf") pod "445167ab-d8fd-4400-aa48-a9223c761008" (UID: "445167ab-d8fd-4400-aa48-a9223c761008"). InnerVolumeSpecName "kube-api-access-xsggf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:45:22.326819 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.326791    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsggf\" (UniqueName: \"kubernetes.io/projected/445167ab-d8fd-4400-aa48-a9223c761008-kube-api-access-xsggf\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:45:22.326819 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.326815    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5wvh\" (UniqueName: \"kubernetes.io/projected/de83f3a7-d894-41be-aa11-911ed1017945-kube-api-access-n5wvh\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:45:22.501498 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.501439    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-659996695f-9jmnm"]
Sep 19 10:45:22.504507 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:45:22.504481    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97d6a09c_8ef5_40f9_841f_a8fee9914bca.slice/crio-eedcd73bbe76abf069637982a718307eb3816d37c522993d041076c70cd72903 WatchSource:0}: Error finding container eedcd73bbe76abf069637982a718307eb3816d37c522993d041076c70cd72903: Status 404 returned error can't find the container with id eedcd73bbe76abf069637982a718307eb3816d37c522993d041076c70cd72903
Sep 19 10:45:22.788944 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.788870    2576 generic.go:356] "Generic (PLEG): container finished" podID="445167ab-d8fd-4400-aa48-a9223c761008" containerID="1bf57ed0ecea0696d0130d6f9b9548b3a435dfa622476c56723bc02c712b0da5" exitCode=0
Sep 19 10:45:22.788944 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.788908    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-lnmq9"
Sep 19 10:45:22.789129 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.788947    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-lnmq9" event={"ID":"445167ab-d8fd-4400-aa48-a9223c761008","Type":"ContainerDied","Data":"1bf57ed0ecea0696d0130d6f9b9548b3a435dfa622476c56723bc02c712b0da5"}
Sep 19 10:45:22.789129 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.788981    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-lnmq9" event={"ID":"445167ab-d8fd-4400-aa48-a9223c761008","Type":"ContainerDied","Data":"6303dcf2b6e4d95cbe997d88541ccd0c84f63b6d16fd6a5aee3ce91335a45483"}
Sep 19 10:45:22.789129 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.788999    2576 scope.go:117] "RemoveContainer" containerID="1bf57ed0ecea0696d0130d6f9b9548b3a435dfa622476c56723bc02c712b0da5"
Sep 19 10:45:22.790427 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.790399    2576 generic.go:356] "Generic (PLEG): container finished" podID="de83f3a7-d894-41be-aa11-911ed1017945" containerID="efb99787fc2888d48ec501e7485f6aef7b4f2207b1ccfc4cd88fb81e99f28d84" exitCode=0
Sep 19 10:45:22.790517 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.790429    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-5lvbt" event={"ID":"de83f3a7-d894-41be-aa11-911ed1017945","Type":"ContainerDied","Data":"efb99787fc2888d48ec501e7485f6aef7b4f2207b1ccfc4cd88fb81e99f28d84"}
Sep 19 10:45:22.790517 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.790480    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-5lvbt" event={"ID":"de83f3a7-d894-41be-aa11-911ed1017945","Type":"ContainerDied","Data":"d02e2399221440be7e8c65eacbe9b9685f6e77039d9f572c147b92e16b8f8b0d"}
Sep 19 10:45:22.790517 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.790443    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-5lvbt"
Sep 19 10:45:22.791865 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.791842    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-659996695f-9jmnm" event={"ID":"97d6a09c-8ef5-40f9-841f-a8fee9914bca","Type":"ContainerStarted","Data":"eedcd73bbe76abf069637982a718307eb3816d37c522993d041076c70cd72903"}
Sep 19 10:45:22.798302 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.798280    2576 scope.go:117] "RemoveContainer" containerID="1bf57ed0ecea0696d0130d6f9b9548b3a435dfa622476c56723bc02c712b0da5"
Sep 19 10:45:22.798676 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:45:22.798613    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bf57ed0ecea0696d0130d6f9b9548b3a435dfa622476c56723bc02c712b0da5\": container with ID starting with 1bf57ed0ecea0696d0130d6f9b9548b3a435dfa622476c56723bc02c712b0da5 not found: ID does not exist" containerID="1bf57ed0ecea0696d0130d6f9b9548b3a435dfa622476c56723bc02c712b0da5"
Sep 19 10:45:22.798777 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.798668    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bf57ed0ecea0696d0130d6f9b9548b3a435dfa622476c56723bc02c712b0da5"} err="failed to get container status \"1bf57ed0ecea0696d0130d6f9b9548b3a435dfa622476c56723bc02c712b0da5\": rpc error: code = NotFound desc = could not find container \"1bf57ed0ecea0696d0130d6f9b9548b3a435dfa622476c56723bc02c712b0da5\": container with ID starting with 1bf57ed0ecea0696d0130d6f9b9548b3a435dfa622476c56723bc02c712b0da5 not found: ID does not exist"
Sep 19 10:45:22.798777 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.798693    2576 scope.go:117] "RemoveContainer" containerID="efb99787fc2888d48ec501e7485f6aef7b4f2207b1ccfc4cd88fb81e99f28d84"
Sep 19 10:45:22.807979 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.807960    2576 scope.go:117] "RemoveContainer" containerID="efb99787fc2888d48ec501e7485f6aef7b4f2207b1ccfc4cd88fb81e99f28d84"
Sep 19 10:45:22.808260 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:45:22.808242    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efb99787fc2888d48ec501e7485f6aef7b4f2207b1ccfc4cd88fb81e99f28d84\": container with ID starting with efb99787fc2888d48ec501e7485f6aef7b4f2207b1ccfc4cd88fb81e99f28d84 not found: ID does not exist" containerID="efb99787fc2888d48ec501e7485f6aef7b4f2207b1ccfc4cd88fb81e99f28d84"
Sep 19 10:45:22.808325 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.808267    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efb99787fc2888d48ec501e7485f6aef7b4f2207b1ccfc4cd88fb81e99f28d84"} err="failed to get container status \"efb99787fc2888d48ec501e7485f6aef7b4f2207b1ccfc4cd88fb81e99f28d84\": rpc error: code = NotFound desc = could not find container \"efb99787fc2888d48ec501e7485f6aef7b4f2207b1ccfc4cd88fb81e99f28d84\": container with ID starting with efb99787fc2888d48ec501e7485f6aef7b4f2207b1ccfc4cd88fb81e99f28d84 not found: ID does not exist"
Sep 19 10:45:22.814445 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.814427    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-lnmq9"]
Sep 19 10:45:22.817983 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.817963    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-lnmq9"]
Sep 19 10:45:22.826905 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.826883    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-5lvbt"]
Sep 19 10:45:22.831398 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:22.831378    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-5lvbt"]
Sep 19 10:45:23.002749 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:23.002722    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="445167ab-d8fd-4400-aa48-a9223c761008" path="/var/lib/kubelet/pods/445167ab-d8fd-4400-aa48-a9223c761008/volumes"
Sep 19 10:45:23.003119 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:23.003101    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de83f3a7-d894-41be-aa11-911ed1017945" path="/var/lib/kubelet/pods/de83f3a7-d894-41be-aa11-911ed1017945/volumes"
Sep 19 10:45:23.798616 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:23.798575    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-659996695f-9jmnm" event={"ID":"97d6a09c-8ef5-40f9-841f-a8fee9914bca","Type":"ContainerStarted","Data":"e6ae8b55cb6340cde572260e2a424bb1ba9b970c747c04a66764d5f452563ad8"}
Sep 19 10:45:23.812432 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:23.812372    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-659996695f-9jmnm" podStartSLOduration=2.263977754 podStartE2EDuration="2.812353305s" totalImagesPullingTime="548.375551ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:45:23.811866391 +0000 UTC m=+533.308172483" watchObservedRunningTime="2026-09-19 10:45:23.812353305 +0000 UTC m=+533.308659398"
Sep 19 10:45:23.835634 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:23.835606    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-799ff98669-fqhdr"]
Sep 19 10:45:23.836068 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:23.835814    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-799ff98669-fqhdr" podUID="63256afe-3f42-4a24-957a-e8a478d7f44e" containerName="authorino" containerID="cri-o://ddac78d74830d437944b87fdf89ea6ea6fd26e88ce1a9e3816cacda31b1ffa0f" gracePeriod=30
Sep 19 10:45:24.112132 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:24.112108    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-799ff98669-fqhdr"
Sep 19 10:45:24.242637 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:24.242609    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-868cw\" (UniqueName: \"kubernetes.io/projected/63256afe-3f42-4a24-957a-e8a478d7f44e-kube-api-access-868cw\") pod \"63256afe-3f42-4a24-957a-e8a478d7f44e\" (UID: \"63256afe-3f42-4a24-957a-e8a478d7f44e\") "
Sep 19 10:45:24.244722 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:24.244700    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63256afe-3f42-4a24-957a-e8a478d7f44e-kube-api-access-868cw" (OuterVolumeSpecName: "kube-api-access-868cw") pod "63256afe-3f42-4a24-957a-e8a478d7f44e" (UID: "63256afe-3f42-4a24-957a-e8a478d7f44e"). InnerVolumeSpecName "kube-api-access-868cw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:45:24.343519 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:24.343497    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-868cw\" (UniqueName: \"kubernetes.io/projected/63256afe-3f42-4a24-957a-e8a478d7f44e-kube-api-access-868cw\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:45:24.806292 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:24.806220    2576 generic.go:356] "Generic (PLEG): container finished" podID="63256afe-3f42-4a24-957a-e8a478d7f44e" containerID="ddac78d74830d437944b87fdf89ea6ea6fd26e88ce1a9e3816cacda31b1ffa0f" exitCode=0
Sep 19 10:45:24.806292 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:24.806263    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-799ff98669-fqhdr"
Sep 19 10:45:24.806453 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:24.806304    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-799ff98669-fqhdr" event={"ID":"63256afe-3f42-4a24-957a-e8a478d7f44e","Type":"ContainerDied","Data":"ddac78d74830d437944b87fdf89ea6ea6fd26e88ce1a9e3816cacda31b1ffa0f"}
Sep 19 10:45:24.806453 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:24.806350    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-799ff98669-fqhdr" event={"ID":"63256afe-3f42-4a24-957a-e8a478d7f44e","Type":"ContainerDied","Data":"c6ba49add8a2a74f4753f32312a93b8e5389f3726223bdb1f57022bafac5a1b8"}
Sep 19 10:45:24.806453 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:24.806369    2576 scope.go:117] "RemoveContainer" containerID="ddac78d74830d437944b87fdf89ea6ea6fd26e88ce1a9e3816cacda31b1ffa0f"
Sep 19 10:45:24.815717 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:24.815698    2576 scope.go:117] "RemoveContainer" containerID="ddac78d74830d437944b87fdf89ea6ea6fd26e88ce1a9e3816cacda31b1ffa0f"
Sep 19 10:45:24.815979 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:45:24.815962    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddac78d74830d437944b87fdf89ea6ea6fd26e88ce1a9e3816cacda31b1ffa0f\": container with ID starting with ddac78d74830d437944b87fdf89ea6ea6fd26e88ce1a9e3816cacda31b1ffa0f not found: ID does not exist" containerID="ddac78d74830d437944b87fdf89ea6ea6fd26e88ce1a9e3816cacda31b1ffa0f"
Sep 19 10:45:24.816066 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:24.815990    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddac78d74830d437944b87fdf89ea6ea6fd26e88ce1a9e3816cacda31b1ffa0f"} err="failed to get container status \"ddac78d74830d437944b87fdf89ea6ea6fd26e88ce1a9e3816cacda31b1ffa0f\": rpc error: code = NotFound desc = could not find container \"ddac78d74830d437944b87fdf89ea6ea6fd26e88ce1a9e3816cacda31b1ffa0f\": container with ID starting with ddac78d74830d437944b87fdf89ea6ea6fd26e88ce1a9e3816cacda31b1ffa0f not found: ID does not exist"
Sep 19 10:45:24.826832 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:24.826809    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-799ff98669-fqhdr"]
Sep 19 10:45:24.830566 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:24.830543    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-799ff98669-fqhdr"]
Sep 19 10:45:25.001292 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:25.001268    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63256afe-3f42-4a24-957a-e8a478d7f44e" path="/var/lib/kubelet/pods/63256afe-3f42-4a24-957a-e8a478d7f44e/volumes"
Sep 19 10:45:26.405118 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.405087    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-778f475c74-splj9"]
Sep 19 10:45:26.405468 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.405449    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="445167ab-d8fd-4400-aa48-a9223c761008" containerName="authorino"
Sep 19 10:45:26.405468 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.405460    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="445167ab-d8fd-4400-aa48-a9223c761008" containerName="authorino"
Sep 19 10:45:26.405468 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.405469    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de83f3a7-d894-41be-aa11-911ed1017945" containerName="authorino"
Sep 19 10:45:26.405564 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.405474    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="de83f3a7-d894-41be-aa11-911ed1017945" containerName="authorino"
Sep 19 10:45:26.405564 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.405481    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63256afe-3f42-4a24-957a-e8a478d7f44e" containerName="authorino"
Sep 19 10:45:26.405564 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.405486    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="63256afe-3f42-4a24-957a-e8a478d7f44e" containerName="authorino"
Sep 19 10:45:26.405564 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.405545    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="445167ab-d8fd-4400-aa48-a9223c761008" containerName="authorino"
Sep 19 10:45:26.405564 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.405552    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="de83f3a7-d894-41be-aa11-911ed1017945" containerName="authorino"
Sep 19 10:45:26.405564 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.405561    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="63256afe-3f42-4a24-957a-e8a478d7f44e" containerName="authorino"
Sep 19 10:45:26.407951 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.407934    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-778f475c74-splj9"
Sep 19 10:45:26.410626 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.410601    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 19 10:45:26.410736 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.410681    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 19 10:45:26.410775 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.410758    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6lwg4\""
Sep 19 10:45:26.410824 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.410760    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 19 10:45:26.417466 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.417446    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-778f475c74-splj9"]
Sep 19 10:45:26.559880 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.559857    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d90a9d4d-82a9-4710-8c56-e519c6b062b5-webhook-cert\") pod \"maas-controller-778f475c74-splj9\" (UID: \"d90a9d4d-82a9-4710-8c56-e519c6b062b5\") " pod="opendatahub/maas-controller-778f475c74-splj9"
Sep 19 10:45:26.560022 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.559901    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvw9l\" (UniqueName: \"kubernetes.io/projected/d90a9d4d-82a9-4710-8c56-e519c6b062b5-kube-api-access-xvw9l\") pod \"maas-controller-778f475c74-splj9\" (UID: \"d90a9d4d-82a9-4710-8c56-e519c6b062b5\") " pod="opendatahub/maas-controller-778f475c74-splj9"
Sep 19 10:45:26.560022 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.559934    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d90a9d4d-82a9-4710-8c56-e519c6b062b5-metrics-certs\") pod \"maas-controller-778f475c74-splj9\" (UID: \"d90a9d4d-82a9-4710-8c56-e519c6b062b5\") " pod="opendatahub/maas-controller-778f475c74-splj9"
Sep 19 10:45:26.579068 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.579027    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7599657744-dccbf"]
Sep 19 10:45:26.581283 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.581270    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7599657744-dccbf"
Sep 19 10:45:26.591283 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.591256    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7599657744-dccbf"]
Sep 19 10:45:26.660672 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.660617    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d90a9d4d-82a9-4710-8c56-e519c6b062b5-webhook-cert\") pod \"maas-controller-778f475c74-splj9\" (UID: \"d90a9d4d-82a9-4710-8c56-e519c6b062b5\") " pod="opendatahub/maas-controller-778f475c74-splj9"
Sep 19 10:45:26.660672 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.660658    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvw9l\" (UniqueName: \"kubernetes.io/projected/d90a9d4d-82a9-4710-8c56-e519c6b062b5-kube-api-access-xvw9l\") pod \"maas-controller-778f475c74-splj9\" (UID: \"d90a9d4d-82a9-4710-8c56-e519c6b062b5\") " pod="opendatahub/maas-controller-778f475c74-splj9"
Sep 19 10:45:26.660793 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.660702    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d90a9d4d-82a9-4710-8c56-e519c6b062b5-metrics-certs\") pod \"maas-controller-778f475c74-splj9\" (UID: \"d90a9d4d-82a9-4710-8c56-e519c6b062b5\") " pod="opendatahub/maas-controller-778f475c74-splj9"
Sep 19 10:45:26.663381 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.663354    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d90a9d4d-82a9-4710-8c56-e519c6b062b5-webhook-cert\") pod \"maas-controller-778f475c74-splj9\" (UID: \"d90a9d4d-82a9-4710-8c56-e519c6b062b5\") " pod="opendatahub/maas-controller-778f475c74-splj9"
Sep 19 10:45:26.663381 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.663377    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d90a9d4d-82a9-4710-8c56-e519c6b062b5-metrics-certs\") pod \"maas-controller-778f475c74-splj9\" (UID: \"d90a9d4d-82a9-4710-8c56-e519c6b062b5\") " pod="opendatahub/maas-controller-778f475c74-splj9"
Sep 19 10:45:26.667796 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.667773    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvw9l\" (UniqueName: \"kubernetes.io/projected/d90a9d4d-82a9-4710-8c56-e519c6b062b5-kube-api-access-xvw9l\") pod \"maas-controller-778f475c74-splj9\" (UID: \"d90a9d4d-82a9-4710-8c56-e519c6b062b5\") " pod="opendatahub/maas-controller-778f475c74-splj9"
Sep 19 10:45:26.719312 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.719290    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-778f475c74-splj9"
Sep 19 10:45:26.761958 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.761931    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06077445-6116-47dc-b23d-ddb2e73db131-metrics-certs\") pod \"maas-controller-7599657744-dccbf\" (UID: \"06077445-6116-47dc-b23d-ddb2e73db131\") " pod="opendatahub/maas-controller-7599657744-dccbf"
Sep 19 10:45:26.762104 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.761965    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5frs\" (UniqueName: \"kubernetes.io/projected/06077445-6116-47dc-b23d-ddb2e73db131-kube-api-access-n5frs\") pod \"maas-controller-7599657744-dccbf\" (UID: \"06077445-6116-47dc-b23d-ddb2e73db131\") " pod="opendatahub/maas-controller-7599657744-dccbf"
Sep 19 10:45:26.762104 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.762036    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06077445-6116-47dc-b23d-ddb2e73db131-webhook-cert\") pod \"maas-controller-7599657744-dccbf\" (UID: \"06077445-6116-47dc-b23d-ddb2e73db131\") " pod="opendatahub/maas-controller-7599657744-dccbf"
Sep 19 10:45:26.863355 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.863332    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06077445-6116-47dc-b23d-ddb2e73db131-metrics-certs\") pod \"maas-controller-7599657744-dccbf\" (UID: \"06077445-6116-47dc-b23d-ddb2e73db131\") " pod="opendatahub/maas-controller-7599657744-dccbf"
Sep 19 10:45:26.863488 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.863363    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5frs\" (UniqueName: \"kubernetes.io/projected/06077445-6116-47dc-b23d-ddb2e73db131-kube-api-access-n5frs\") pod \"maas-controller-7599657744-dccbf\" (UID: \"06077445-6116-47dc-b23d-ddb2e73db131\") " pod="opendatahub/maas-controller-7599657744-dccbf"
Sep 19 10:45:26.863488 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.863406    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06077445-6116-47dc-b23d-ddb2e73db131-webhook-cert\") pod \"maas-controller-7599657744-dccbf\" (UID: \"06077445-6116-47dc-b23d-ddb2e73db131\") " pod="opendatahub/maas-controller-7599657744-dccbf"
Sep 19 10:45:26.865823 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.865794    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06077445-6116-47dc-b23d-ddb2e73db131-metrics-certs\") pod \"maas-controller-7599657744-dccbf\" (UID: \"06077445-6116-47dc-b23d-ddb2e73db131\") " pod="opendatahub/maas-controller-7599657744-dccbf"
Sep 19 10:45:26.865990 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.865967    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06077445-6116-47dc-b23d-ddb2e73db131-webhook-cert\") pod \"maas-controller-7599657744-dccbf\" (UID: \"06077445-6116-47dc-b23d-ddb2e73db131\") " pod="opendatahub/maas-controller-7599657744-dccbf"
Sep 19 10:45:26.871025 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.871006    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5frs\" (UniqueName: \"kubernetes.io/projected/06077445-6116-47dc-b23d-ddb2e73db131-kube-api-access-n5frs\") pod \"maas-controller-7599657744-dccbf\" (UID: \"06077445-6116-47dc-b23d-ddb2e73db131\") " pod="opendatahub/maas-controller-7599657744-dccbf"
Sep 19 10:45:26.890976 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.890951    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7599657744-dccbf"
Sep 19 10:45:26.987673 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:26.987644    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-778f475c74-splj9"]
Sep 19 10:45:26.989483 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:45:26.989459    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd90a9d4d_82a9_4710_8c56_e519c6b062b5.slice/crio-df3c971fc634dcac4e5da24b374948f8021235ad97abec03665506e128664fe7 WatchSource:0}: Error finding container df3c971fc634dcac4e5da24b374948f8021235ad97abec03665506e128664fe7: Status 404 returned error can't find the container with id df3c971fc634dcac4e5da24b374948f8021235ad97abec03665506e128664fe7
Sep 19 10:45:27.156456 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:27.156435    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7599657744-dccbf"]
Sep 19 10:45:27.158201 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:45:27.158177    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06077445_6116_47dc_b23d_ddb2e73db131.slice/crio-4030db093a0cef1743f4f51c4add2154893ab6c57d6b0cfc2af8a39d85e3ebac WatchSource:0}: Error finding container 4030db093a0cef1743f4f51c4add2154893ab6c57d6b0cfc2af8a39d85e3ebac: Status 404 returned error can't find the container with id 4030db093a0cef1743f4f51c4add2154893ab6c57d6b0cfc2af8a39d85e3ebac
Sep 19 10:45:27.829603 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:27.829518    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7599657744-dccbf" event={"ID":"06077445-6116-47dc-b23d-ddb2e73db131","Type":"ContainerStarted","Data":"4030db093a0cef1743f4f51c4add2154893ab6c57d6b0cfc2af8a39d85e3ebac"}
Sep 19 10:45:27.831496 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:27.831437    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-778f475c74-splj9" event={"ID":"d90a9d4d-82a9-4710-8c56-e519c6b062b5","Type":"ContainerStarted","Data":"df3c971fc634dcac4e5da24b374948f8021235ad97abec03665506e128664fe7"}
Sep 19 10:45:29.843833 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:29.843749    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-778f475c74-splj9" event={"ID":"d90a9d4d-82a9-4710-8c56-e519c6b062b5","Type":"ContainerStarted","Data":"3780b1227ff1a0c9890076a334e77fce8e2bda41d4d00e607cdca8199c110bd9"}
Sep 19 10:45:29.843833 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:29.843824    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-778f475c74-splj9"
Sep 19 10:45:29.845264 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:29.845240    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7599657744-dccbf" event={"ID":"06077445-6116-47dc-b23d-ddb2e73db131","Type":"ContainerStarted","Data":"b6ade56169226ab6c92f778f1f8f257bcb82dbbf97c1eb8cf2e97d996424a99c"}
Sep 19 10:45:29.845413 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:29.845364    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7599657744-dccbf"
Sep 19 10:45:29.873266 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:29.873222    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-778f475c74-splj9" podStartSLOduration=1.271583754 podStartE2EDuration="3.873210228s" totalImagesPullingTime="2.601626474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:45:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:45:29.871446281 +0000 UTC m=+539.367752372" watchObservedRunningTime="2026-09-19 10:45:29.873210228 +0000 UTC m=+539.369516320"
Sep 19 10:45:29.902867 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:29.902829    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7599657744-dccbf" podStartSLOduration=1.460002673 podStartE2EDuration="3.902817757s" totalImagesPullingTime="2.442815084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:45:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:45:29.90154842 +0000 UTC m=+539.397854513" watchObservedRunningTime="2026-09-19 10:45:29.902817757 +0000 UTC m=+539.399123849"
Sep 19 10:45:40.852109 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:40.852078    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7599657744-dccbf"
Sep 19 10:45:40.852504 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:40.852148    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-778f475c74-splj9"
Sep 19 10:45:40.916271 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:40.916238    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-778f475c74-splj9"]
Sep 19 10:45:40.916537 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:40.916492    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-778f475c74-splj9" podUID="d90a9d4d-82a9-4710-8c56-e519c6b062b5" containerName="manager" containerID="cri-o://3780b1227ff1a0c9890076a334e77fce8e2bda41d4d00e607cdca8199c110bd9" gracePeriod=30
Sep 19 10:45:41.163088 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:41.163065    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-778f475c74-splj9"
Sep 19 10:45:41.185421 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:41.185397    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d90a9d4d-82a9-4710-8c56-e519c6b062b5-webhook-cert\") pod \"d90a9d4d-82a9-4710-8c56-e519c6b062b5\" (UID: \"d90a9d4d-82a9-4710-8c56-e519c6b062b5\") "
Sep 19 10:45:41.185517 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:41.185458    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d90a9d4d-82a9-4710-8c56-e519c6b062b5-metrics-certs\") pod \"d90a9d4d-82a9-4710-8c56-e519c6b062b5\" (UID: \"d90a9d4d-82a9-4710-8c56-e519c6b062b5\") "
Sep 19 10:45:41.185517 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:41.185496    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvw9l\" (UniqueName: \"kubernetes.io/projected/d90a9d4d-82a9-4710-8c56-e519c6b062b5-kube-api-access-xvw9l\") pod \"d90a9d4d-82a9-4710-8c56-e519c6b062b5\" (UID: \"d90a9d4d-82a9-4710-8c56-e519c6b062b5\") "
Sep 19 10:45:41.187742 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:41.187711    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d90a9d4d-82a9-4710-8c56-e519c6b062b5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d90a9d4d-82a9-4710-8c56-e519c6b062b5" (UID: "d90a9d4d-82a9-4710-8c56-e519c6b062b5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:45:41.187822 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:41.187765    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d90a9d4d-82a9-4710-8c56-e519c6b062b5-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d90a9d4d-82a9-4710-8c56-e519c6b062b5" (UID: "d90a9d4d-82a9-4710-8c56-e519c6b062b5"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:45:41.187822 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:41.187781    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d90a9d4d-82a9-4710-8c56-e519c6b062b5-kube-api-access-xvw9l" (OuterVolumeSpecName: "kube-api-access-xvw9l") pod "d90a9d4d-82a9-4710-8c56-e519c6b062b5" (UID: "d90a9d4d-82a9-4710-8c56-e519c6b062b5"). InnerVolumeSpecName "kube-api-access-xvw9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:45:41.286485 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:41.286460    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d90a9d4d-82a9-4710-8c56-e519c6b062b5-metrics-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:45:41.286485 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:41.286482    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvw9l\" (UniqueName: \"kubernetes.io/projected/d90a9d4d-82a9-4710-8c56-e519c6b062b5-kube-api-access-xvw9l\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:45:41.286600 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:41.286491    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d90a9d4d-82a9-4710-8c56-e519c6b062b5-webhook-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:45:41.911921 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:41.911882    2576 generic.go:356] "Generic (PLEG): container finished" podID="d90a9d4d-82a9-4710-8c56-e519c6b062b5" containerID="3780b1227ff1a0c9890076a334e77fce8e2bda41d4d00e607cdca8199c110bd9" exitCode=0
Sep 19 10:45:41.912414 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:41.911934    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-778f475c74-splj9"
Sep 19 10:45:41.912414 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:41.911976    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-778f475c74-splj9" event={"ID":"d90a9d4d-82a9-4710-8c56-e519c6b062b5","Type":"ContainerDied","Data":"3780b1227ff1a0c9890076a334e77fce8e2bda41d4d00e607cdca8199c110bd9"}
Sep 19 10:45:41.912414 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:41.912062    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-778f475c74-splj9" event={"ID":"d90a9d4d-82a9-4710-8c56-e519c6b062b5","Type":"ContainerDied","Data":"df3c971fc634dcac4e5da24b374948f8021235ad97abec03665506e128664fe7"}
Sep 19 10:45:41.912414 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:41.912106    2576 scope.go:117] "RemoveContainer" containerID="3780b1227ff1a0c9890076a334e77fce8e2bda41d4d00e607cdca8199c110bd9"
Sep 19 10:45:41.922685 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:41.922481    2576 scope.go:117] "RemoveContainer" containerID="3780b1227ff1a0c9890076a334e77fce8e2bda41d4d00e607cdca8199c110bd9"
Sep 19 10:45:41.922815 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:45:41.922792    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3780b1227ff1a0c9890076a334e77fce8e2bda41d4d00e607cdca8199c110bd9\": container with ID starting with 3780b1227ff1a0c9890076a334e77fce8e2bda41d4d00e607cdca8199c110bd9 not found: ID does not exist" containerID="3780b1227ff1a0c9890076a334e77fce8e2bda41d4d00e607cdca8199c110bd9"
Sep 19 10:45:41.922855 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:41.922822    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3780b1227ff1a0c9890076a334e77fce8e2bda41d4d00e607cdca8199c110bd9"} err="failed to get container status \"3780b1227ff1a0c9890076a334e77fce8e2bda41d4d00e607cdca8199c110bd9\": rpc error: code = NotFound desc = could not find container \"3780b1227ff1a0c9890076a334e77fce8e2bda41d4d00e607cdca8199c110bd9\": container with ID starting with 3780b1227ff1a0c9890076a334e77fce8e2bda41d4d00e607cdca8199c110bd9 not found: ID does not exist"
Sep 19 10:45:41.933364 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:41.933342    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-778f475c74-splj9"]
Sep 19 10:45:41.937390 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:41.937369    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-778f475c74-splj9"]
Sep 19 10:45:43.000468 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:45:43.000441    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d90a9d4d-82a9-4710-8c56-e519c6b062b5" path="/var/lib/kubelet/pods/d90a9d4d-82a9-4710-8c56-e519c6b062b5/volumes"
Sep 19 10:46:02.937973 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:02.937941    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"]
Sep 19 10:46:02.938420 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:02.938363    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d90a9d4d-82a9-4710-8c56-e519c6b062b5" containerName="manager"
Sep 19 10:46:02.938420 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:02.938376    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d90a9d4d-82a9-4710-8c56-e519c6b062b5" containerName="manager"
Sep 19 10:46:02.938492 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:02.938432    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d90a9d4d-82a9-4710-8c56-e519c6b062b5" containerName="manager"
Sep 19 10:46:02.943769 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:02.943751    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"
Sep 19 10:46:02.946204 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:02.946180    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 19 10:46:02.946204 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:02.946196    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g2pqv\""
Sep 19 10:46:02.946362 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:02.946195    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 19 10:46:02.951586 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:02.951518    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"]
Sep 19 10:46:03.050363 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.050330    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1982fcfa-c5e2-42b7-8cd4-4f08da150620-openshift-service-ca\") pod \"maas-api-6599cb9dcd-pr4wz\" (UID: \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\") " pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"
Sep 19 10:46:03.050539 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.050378    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1982fcfa-c5e2-42b7-8cd4-4f08da150620-metrics-certs\") pod \"maas-api-6599cb9dcd-pr4wz\" (UID: \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\") " pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"
Sep 19 10:46:03.050539 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.050412    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1982fcfa-c5e2-42b7-8cd4-4f08da150620-maas-api-tls\") pod \"maas-api-6599cb9dcd-pr4wz\" (UID: \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\") " pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"
Sep 19 10:46:03.050539 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.050507    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd567\" (UniqueName: \"kubernetes.io/projected/1982fcfa-c5e2-42b7-8cd4-4f08da150620-kube-api-access-xd567\") pod \"maas-api-6599cb9dcd-pr4wz\" (UID: \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\") " pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"
Sep 19 10:46:03.151525 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.151495    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1982fcfa-c5e2-42b7-8cd4-4f08da150620-openshift-service-ca\") pod \"maas-api-6599cb9dcd-pr4wz\" (UID: \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\") " pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"
Sep 19 10:46:03.151651 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.151542    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1982fcfa-c5e2-42b7-8cd4-4f08da150620-metrics-certs\") pod \"maas-api-6599cb9dcd-pr4wz\" (UID: \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\") " pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"
Sep 19 10:46:03.151651 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.151574    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1982fcfa-c5e2-42b7-8cd4-4f08da150620-maas-api-tls\") pod \"maas-api-6599cb9dcd-pr4wz\" (UID: \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\") " pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"
Sep 19 10:46:03.151651 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.151607    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd567\" (UniqueName: \"kubernetes.io/projected/1982fcfa-c5e2-42b7-8cd4-4f08da150620-kube-api-access-xd567\") pod \"maas-api-6599cb9dcd-pr4wz\" (UID: \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\") " pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"
Sep 19 10:46:03.152157 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.152137    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1982fcfa-c5e2-42b7-8cd4-4f08da150620-openshift-service-ca\") pod \"maas-api-6599cb9dcd-pr4wz\" (UID: \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\") " pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"
Sep 19 10:46:03.153975 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.153950    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1982fcfa-c5e2-42b7-8cd4-4f08da150620-metrics-certs\") pod \"maas-api-6599cb9dcd-pr4wz\" (UID: \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\") " pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"
Sep 19 10:46:03.154118 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.153992    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1982fcfa-c5e2-42b7-8cd4-4f08da150620-maas-api-tls\") pod \"maas-api-6599cb9dcd-pr4wz\" (UID: \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\") " pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"
Sep 19 10:46:03.159197 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.159177    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd567\" (UniqueName: \"kubernetes.io/projected/1982fcfa-c5e2-42b7-8cd4-4f08da150620-kube-api-access-xd567\") pod \"maas-api-6599cb9dcd-pr4wz\" (UID: \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\") " pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"
Sep 19 10:46:03.256411 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.256355    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"
Sep 19 10:46:03.474095 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.474057    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-fm8xm"]
Sep 19 10:46:03.478258 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.478242    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fm8xm"
Sep 19 10:46:03.480653 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.480636    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 19 10:46:03.488575 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.488554    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-fm8xm"]
Sep 19 10:46:03.533640 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.533615    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"]
Sep 19 10:46:03.535680 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:46:03.535646    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1982fcfa_c5e2_42b7_8cd4_4f08da150620.slice/crio-fdb6c3de975aee712ece39729efc57bfc843ff8806b59ffebaa570d09e713488 WatchSource:0}: Error finding container fdb6c3de975aee712ece39729efc57bfc843ff8806b59ffebaa570d09e713488: Status 404 returned error can't find the container with id fdb6c3de975aee712ece39729efc57bfc843ff8806b59ffebaa570d09e713488
Sep 19 10:46:03.555968 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.555944    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mfhl\" (UniqueName: \"kubernetes.io/projected/fb66578d-ea20-470f-83dc-0c84c6d9ae7e-kube-api-access-8mfhl\") pod \"payload-pre-processing-59fbc47bf8-fm8xm\" (UID: \"fb66578d-ea20-470f-83dc-0c84c6d9ae7e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fm8xm"
Sep 19 10:46:03.556085 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.556014    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb66578d-ea20-470f-83dc-0c84c6d9ae7e-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-fm8xm\" (UID: \"fb66578d-ea20-470f-83dc-0c84c6d9ae7e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fm8xm"
Sep 19 10:46:03.657279 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.657244    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mfhl\" (UniqueName: \"kubernetes.io/projected/fb66578d-ea20-470f-83dc-0c84c6d9ae7e-kube-api-access-8mfhl\") pod \"payload-pre-processing-59fbc47bf8-fm8xm\" (UID: \"fb66578d-ea20-470f-83dc-0c84c6d9ae7e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fm8xm"
Sep 19 10:46:03.657474 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.657331    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb66578d-ea20-470f-83dc-0c84c6d9ae7e-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-fm8xm\" (UID: \"fb66578d-ea20-470f-83dc-0c84c6d9ae7e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fm8xm"
Sep 19 10:46:03.658129 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.658108    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb66578d-ea20-470f-83dc-0c84c6d9ae7e-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-fm8xm\" (UID: \"fb66578d-ea20-470f-83dc-0c84c6d9ae7e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fm8xm"
Sep 19 10:46:03.668188 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.668165    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mfhl\" (UniqueName: \"kubernetes.io/projected/fb66578d-ea20-470f-83dc-0c84c6d9ae7e-kube-api-access-8mfhl\") pod \"payload-pre-processing-59fbc47bf8-fm8xm\" (UID: \"fb66578d-ea20-470f-83dc-0c84c6d9ae7e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fm8xm"
Sep 19 10:46:03.790017 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:03.789917    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fm8xm"
Sep 19 10:46:04.020968 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:04.020935    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz" event={"ID":"1982fcfa-c5e2-42b7-8cd4-4f08da150620","Type":"ContainerStarted","Data":"fdb6c3de975aee712ece39729efc57bfc843ff8806b59ffebaa570d09e713488"}
Sep 19 10:46:04.074499 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:04.074475    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-fm8xm"]
Sep 19 10:46:04.076558 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:46:04.076533    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb66578d_ea20_470f_83dc_0c84c6d9ae7e.slice/crio-7f1b8472b2f5c9ef3a48df70bdcf9644cdbdca63c26e7a164aa2cda3560c687d WatchSource:0}: Error finding container 7f1b8472b2f5c9ef3a48df70bdcf9644cdbdca63c26e7a164aa2cda3560c687d: Status 404 returned error can't find the container with id 7f1b8472b2f5c9ef3a48df70bdcf9644cdbdca63c26e7a164aa2cda3560c687d
Sep 19 10:46:05.028710 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:05.028644    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fm8xm" event={"ID":"fb66578d-ea20-470f-83dc-0c84c6d9ae7e","Type":"ContainerStarted","Data":"7f1b8472b2f5c9ef3a48df70bdcf9644cdbdca63c26e7a164aa2cda3560c687d"}
Sep 19 10:46:06.035645 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:06.035615    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz" event={"ID":"1982fcfa-c5e2-42b7-8cd4-4f08da150620","Type":"ContainerStarted","Data":"1dec620c6ef9bf8904072c064c06b3540db2ced662ff59d7469f2a484d94668b"}
Sep 19 10:46:06.036173 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:06.035689    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"
Sep 19 10:46:06.052779 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:06.052713    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz" podStartSLOduration=2.471305599 podStartE2EDuration="4.052697538s" totalImagesPullingTime="1.581391939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:46:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:46:06.051818263 +0000 UTC m=+575.548124358" watchObservedRunningTime="2026-09-19 10:46:06.052697538 +0000 UTC m=+575.549003630"
Sep 19 10:46:07.041946 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:07.041909    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fm8xm" event={"ID":"fb66578d-ea20-470f-83dc-0c84c6d9ae7e","Type":"ContainerStarted","Data":"df2d5f0806ddb5967c81cc77a03c9be3b8c3bd84a97d5f7b9233a3f0ee518afb"}
Sep 19 10:46:07.057348 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:07.057305    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fm8xm" podStartSLOduration=1.725351986 podStartE2EDuration="4.057290444s" totalImagesPullingTime="2.331938458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:46:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:46:07.056160712 +0000 UTC m=+576.552466804" watchObservedRunningTime="2026-09-19 10:46:07.057290444 +0000 UTC m=+576.553596536"
Sep 19 10:46:08.047849 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:08.047813    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fm8xm"
Sep 19 10:46:12.047549 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:12.047514    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"
Sep 19 10:46:19.054630 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:19.054601    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-fm8xm"
Sep 19 10:46:29.013134 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.013096    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"]
Sep 19 10:46:29.016872 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.016849    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.019307 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.019283    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 19 10:46:29.019413 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.019307    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 19 10:46:29.020198 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.020182    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 19 10:46:29.020292 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.020212    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-prjcd\""
Sep 19 10:46:29.025946 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.025925    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"]
Sep 19 10:46:29.185442 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.185400    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33522469-58db-4187-a1d2-aa13594dc9c9-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj\" (UID: \"33522469-58db-4187-a1d2-aa13594dc9c9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.185442 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.185440    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33522469-58db-4187-a1d2-aa13594dc9c9-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj\" (UID: \"33522469-58db-4187-a1d2-aa13594dc9c9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.185675 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.185470    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33522469-58db-4187-a1d2-aa13594dc9c9-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj\" (UID: \"33522469-58db-4187-a1d2-aa13594dc9c9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.185675 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.185599    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33522469-58db-4187-a1d2-aa13594dc9c9-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj\" (UID: \"33522469-58db-4187-a1d2-aa13594dc9c9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.185675 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.185622    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33522469-58db-4187-a1d2-aa13594dc9c9-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj\" (UID: \"33522469-58db-4187-a1d2-aa13594dc9c9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.185675 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.185647    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glrt8\" (UniqueName: \"kubernetes.io/projected/33522469-58db-4187-a1d2-aa13594dc9c9-kube-api-access-glrt8\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj\" (UID: \"33522469-58db-4187-a1d2-aa13594dc9c9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.286727 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.286660    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33522469-58db-4187-a1d2-aa13594dc9c9-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj\" (UID: \"33522469-58db-4187-a1d2-aa13594dc9c9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.286727 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.286694    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33522469-58db-4187-a1d2-aa13594dc9c9-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj\" (UID: \"33522469-58db-4187-a1d2-aa13594dc9c9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.286727 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.286711    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glrt8\" (UniqueName: \"kubernetes.io/projected/33522469-58db-4187-a1d2-aa13594dc9c9-kube-api-access-glrt8\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj\" (UID: \"33522469-58db-4187-a1d2-aa13594dc9c9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.286967 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.286746    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33522469-58db-4187-a1d2-aa13594dc9c9-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj\" (UID: \"33522469-58db-4187-a1d2-aa13594dc9c9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.286967 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.286761    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33522469-58db-4187-a1d2-aa13594dc9c9-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj\" (UID: \"33522469-58db-4187-a1d2-aa13594dc9c9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.286967 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.286816    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33522469-58db-4187-a1d2-aa13594dc9c9-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj\" (UID: \"33522469-58db-4187-a1d2-aa13594dc9c9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.287148 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.287083    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33522469-58db-4187-a1d2-aa13594dc9c9-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj\" (UID: \"33522469-58db-4187-a1d2-aa13594dc9c9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.287148 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.287138    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33522469-58db-4187-a1d2-aa13594dc9c9-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj\" (UID: \"33522469-58db-4187-a1d2-aa13594dc9c9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.287245 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.287186    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33522469-58db-4187-a1d2-aa13594dc9c9-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj\" (UID: \"33522469-58db-4187-a1d2-aa13594dc9c9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.289356 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.289333    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33522469-58db-4187-a1d2-aa13594dc9c9-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj\" (UID: \"33522469-58db-4187-a1d2-aa13594dc9c9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.289468 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.289416    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33522469-58db-4187-a1d2-aa13594dc9c9-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj\" (UID: \"33522469-58db-4187-a1d2-aa13594dc9c9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.293502 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.293484    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glrt8\" (UniqueName: \"kubernetes.io/projected/33522469-58db-4187-a1d2-aa13594dc9c9-kube-api-access-glrt8\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj\" (UID: \"33522469-58db-4187-a1d2-aa13594dc9c9\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.327373 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.327350    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:29.601619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:29.601596    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"]
Sep 19 10:46:29.603513 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:46:29.603476    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33522469_58db_4187_a1d2_aa13594dc9c9.slice/crio-c29d761e6880f2229a58d0b8d588bd6da6de3cd770948d9e9dae837455542b4f WatchSource:0}: Error finding container c29d761e6880f2229a58d0b8d588bd6da6de3cd770948d9e9dae837455542b4f: Status 404 returned error can't find the container with id c29d761e6880f2229a58d0b8d588bd6da6de3cd770948d9e9dae837455542b4f
Sep 19 10:46:30.161949 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:30.161856    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj" event={"ID":"33522469-58db-4187-a1d2-aa13594dc9c9","Type":"ContainerStarted","Data":"c29d761e6880f2229a58d0b8d588bd6da6de3cd770948d9e9dae837455542b4f"}
Sep 19 10:46:30.921894 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:30.921867    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 10:46:30.922912 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:30.922884    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 10:46:31.167243 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:31.167208    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj" event={"ID":"33522469-58db-4187-a1d2-aa13594dc9c9","Type":"ContainerStarted","Data":"c9ec98316a5706eb48536a5a0b43cd78937327750ddfa5253f573c2af97a917c"}
Sep 19 10:46:31.183056 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:31.183006    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj" podStartSLOduration=2.094606649 podStartE2EDuration="3.182990775s" totalImagesPullingTime="1.088384126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:46:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:46:31.182185153 +0000 UTC m=+600.678491245" watchObservedRunningTime="2026-09-19 10:46:31.182990775 +0000 UTC m=+600.679296866"
Sep 19 10:46:39.327559 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:39.327525    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:39.328177 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:39.327632    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:39.339853 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:39.339835    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:46:40.221553 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:46:40.221526    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj"
Sep 19 10:48:19.307791 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:19.307756    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-656bb6cb6-lpwbh"]
Sep 19 10:48:19.311803 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:19.311782    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-656bb6cb6-lpwbh"
Sep 19 10:48:19.320845 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:19.320819    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-656bb6cb6-lpwbh"]
Sep 19 10:48:19.321361 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:19.321339    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78w85\" (UniqueName: \"kubernetes.io/projected/6127ea09-2783-4ecd-9c08-8d6814565859-kube-api-access-78w85\") pod \"authorino-656bb6cb6-lpwbh\" (UID: \"6127ea09-2783-4ecd-9c08-8d6814565859\") " pod="kuadrant-system/authorino-656bb6cb6-lpwbh"
Sep 19 10:48:19.321455 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:19.321382    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6127ea09-2783-4ecd-9c08-8d6814565859-tls-cert\") pod \"authorino-656bb6cb6-lpwbh\" (UID: \"6127ea09-2783-4ecd-9c08-8d6814565859\") " pod="kuadrant-system/authorino-656bb6cb6-lpwbh"
Sep 19 10:48:19.421918 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:19.421893    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78w85\" (UniqueName: \"kubernetes.io/projected/6127ea09-2783-4ecd-9c08-8d6814565859-kube-api-access-78w85\") pod \"authorino-656bb6cb6-lpwbh\" (UID: \"6127ea09-2783-4ecd-9c08-8d6814565859\") " pod="kuadrant-system/authorino-656bb6cb6-lpwbh"
Sep 19 10:48:19.422030 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:19.421935    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6127ea09-2783-4ecd-9c08-8d6814565859-tls-cert\") pod \"authorino-656bb6cb6-lpwbh\" (UID: \"6127ea09-2783-4ecd-9c08-8d6814565859\") " pod="kuadrant-system/authorino-656bb6cb6-lpwbh"
Sep 19 10:48:19.424589 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:19.424566    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6127ea09-2783-4ecd-9c08-8d6814565859-tls-cert\") pod \"authorino-656bb6cb6-lpwbh\" (UID: \"6127ea09-2783-4ecd-9c08-8d6814565859\") " pod="kuadrant-system/authorino-656bb6cb6-lpwbh"
Sep 19 10:48:19.429384 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:19.429361    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78w85\" (UniqueName: \"kubernetes.io/projected/6127ea09-2783-4ecd-9c08-8d6814565859-kube-api-access-78w85\") pod \"authorino-656bb6cb6-lpwbh\" (UID: \"6127ea09-2783-4ecd-9c08-8d6814565859\") " pod="kuadrant-system/authorino-656bb6cb6-lpwbh"
Sep 19 10:48:19.622014 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:19.621990    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-656bb6cb6-lpwbh"
Sep 19 10:48:19.891686 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:19.891652    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-656bb6cb6-lpwbh"]
Sep 19 10:48:19.893574 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:48:19.893548    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6127ea09_2783_4ecd_9c08_8d6814565859.slice/crio-4812d1fae001ce6e3a660e8c8420bccec0103bf313a9e1e6c603e5251c648182 WatchSource:0}: Error finding container 4812d1fae001ce6e3a660e8c8420bccec0103bf313a9e1e6c603e5251c648182: Status 404 returned error can't find the container with id 4812d1fae001ce6e3a660e8c8420bccec0103bf313a9e1e6c603e5251c648182
Sep 19 10:48:19.895264 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:19.895246    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 10:48:20.700143 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:20.700103    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-656bb6cb6-lpwbh" event={"ID":"6127ea09-2783-4ecd-9c08-8d6814565859","Type":"ContainerStarted","Data":"4812d1fae001ce6e3a660e8c8420bccec0103bf313a9e1e6c603e5251c648182"}
Sep 19 10:48:21.711908 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:21.711813    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-656bb6cb6-lpwbh" event={"ID":"6127ea09-2783-4ecd-9c08-8d6814565859","Type":"ContainerStarted","Data":"084558777bb4ce69efccc1048017b9488df4b829e69ec22f156679cbaa5c4289"}
Sep 19 10:48:21.750070 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:21.749991    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-656bb6cb6-lpwbh" podStartSLOduration=1.9125627440000001 podStartE2EDuration="2.749974885s" totalImagesPullingTime="837.412141ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:48:21.747837803 +0000 UTC m=+711.244143894" watchObservedRunningTime="2026-09-19 10:48:21.749974885 +0000 UTC m=+711.246281026"
Sep 19 10:48:21.811818 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:21.811785    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-659996695f-9jmnm"]
Sep 19 10:48:21.812024 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:21.811999    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-659996695f-9jmnm" podUID="97d6a09c-8ef5-40f9-841f-a8fee9914bca" containerName="authorino" containerID="cri-o://e6ae8b55cb6340cde572260e2a424bb1ba9b970c747c04a66764d5f452563ad8" gracePeriod=30
Sep 19 10:48:22.069218 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:22.069193    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-659996695f-9jmnm"
Sep 19 10:48:22.145390 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:22.145358    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97d6a09c-8ef5-40f9-841f-a8fee9914bca-tls-cert\") pod \"97d6a09c-8ef5-40f9-841f-a8fee9914bca\" (UID: \"97d6a09c-8ef5-40f9-841f-a8fee9914bca\") "
Sep 19 10:48:22.145390 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:22.145396    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4t42d\" (UniqueName: \"kubernetes.io/projected/97d6a09c-8ef5-40f9-841f-a8fee9914bca-kube-api-access-4t42d\") pod \"97d6a09c-8ef5-40f9-841f-a8fee9914bca\" (UID: \"97d6a09c-8ef5-40f9-841f-a8fee9914bca\") "
Sep 19 10:48:22.147631 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:22.147597    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97d6a09c-8ef5-40f9-841f-a8fee9914bca-kube-api-access-4t42d" (OuterVolumeSpecName: "kube-api-access-4t42d") pod "97d6a09c-8ef5-40f9-841f-a8fee9914bca" (UID: "97d6a09c-8ef5-40f9-841f-a8fee9914bca"). InnerVolumeSpecName "kube-api-access-4t42d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:48:22.157115 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:22.157085    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97d6a09c-8ef5-40f9-841f-a8fee9914bca-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "97d6a09c-8ef5-40f9-841f-a8fee9914bca" (UID: "97d6a09c-8ef5-40f9-841f-a8fee9914bca"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:48:22.246702 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:22.246617    2576 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/97d6a09c-8ef5-40f9-841f-a8fee9914bca-tls-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:48:22.246702 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:22.246661    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4t42d\" (UniqueName: \"kubernetes.io/projected/97d6a09c-8ef5-40f9-841f-a8fee9914bca-kube-api-access-4t42d\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:48:22.718927 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:22.718888    2576 generic.go:356] "Generic (PLEG): container finished" podID="97d6a09c-8ef5-40f9-841f-a8fee9914bca" containerID="e6ae8b55cb6340cde572260e2a424bb1ba9b970c747c04a66764d5f452563ad8" exitCode=0
Sep 19 10:48:22.718927 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:22.718924    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-659996695f-9jmnm"
Sep 19 10:48:22.719472 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:22.718974    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-659996695f-9jmnm" event={"ID":"97d6a09c-8ef5-40f9-841f-a8fee9914bca","Type":"ContainerDied","Data":"e6ae8b55cb6340cde572260e2a424bb1ba9b970c747c04a66764d5f452563ad8"}
Sep 19 10:48:22.719472 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:22.719020    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-659996695f-9jmnm" event={"ID":"97d6a09c-8ef5-40f9-841f-a8fee9914bca","Type":"ContainerDied","Data":"eedcd73bbe76abf069637982a718307eb3816d37c522993d041076c70cd72903"}
Sep 19 10:48:22.719472 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:22.719053    2576 scope.go:117] "RemoveContainer" containerID="e6ae8b55cb6340cde572260e2a424bb1ba9b970c747c04a66764d5f452563ad8"
Sep 19 10:48:22.728677 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:22.728658    2576 scope.go:117] "RemoveContainer" containerID="e6ae8b55cb6340cde572260e2a424bb1ba9b970c747c04a66764d5f452563ad8"
Sep 19 10:48:22.728953 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:48:22.728937    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6ae8b55cb6340cde572260e2a424bb1ba9b970c747c04a66764d5f452563ad8\": container with ID starting with e6ae8b55cb6340cde572260e2a424bb1ba9b970c747c04a66764d5f452563ad8 not found: ID does not exist" containerID="e6ae8b55cb6340cde572260e2a424bb1ba9b970c747c04a66764d5f452563ad8"
Sep 19 10:48:22.729011 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:22.728960    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6ae8b55cb6340cde572260e2a424bb1ba9b970c747c04a66764d5f452563ad8"} err="failed to get container status \"e6ae8b55cb6340cde572260e2a424bb1ba9b970c747c04a66764d5f452563ad8\": rpc error: code = NotFound desc = could not find container \"e6ae8b55cb6340cde572260e2a424bb1ba9b970c747c04a66764d5f452563ad8\": container with ID starting with e6ae8b55cb6340cde572260e2a424bb1ba9b970c747c04a66764d5f452563ad8 not found: ID does not exist"
Sep 19 10:48:22.739611 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:22.739589    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-659996695f-9jmnm"]
Sep 19 10:48:22.745976 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:22.745955    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-659996695f-9jmnm"]
Sep 19 10:48:23.006712 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:23.006634    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97d6a09c-8ef5-40f9-841f-a8fee9914bca" path="/var/lib/kubelet/pods/97d6a09c-8ef5-40f9-841f-a8fee9914bca/volumes"
Sep 19 10:48:39.251092 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.251059    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"]
Sep 19 10:48:39.251533 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.251458    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97d6a09c-8ef5-40f9-841f-a8fee9914bca" containerName="authorino"
Sep 19 10:48:39.251533 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.251470    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="97d6a09c-8ef5-40f9-841f-a8fee9914bca" containerName="authorino"
Sep 19 10:48:39.251607 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.251538    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="97d6a09c-8ef5-40f9-841f-a8fee9914bca" containerName="authorino"
Sep 19 10:48:39.255950 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.255931    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.258490 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.258459    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-1907c1-openshift-default-dockercfg-l6mlf\""
Sep 19 10:48:39.258596 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.258521    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 19 10:48:39.267754 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.267735    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"]
Sep 19 10:48:39.375221 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.375192    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"]
Sep 19 10:48:39.378977 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.378954    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.381348 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.381323    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-6a0d36-openshift-default-dockercfg-qwx9x\""
Sep 19 10:48:39.387165 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.387114    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-envoy\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.387250 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.387169    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-credential-socket\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.387250 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.387200    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istiod-ca-cert\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.387411 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.387359    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-podinfo\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.387500 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.387442    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb64f\" (UniqueName: \"kubernetes.io/projected/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-kube-api-access-wb64f\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.387611 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.387593    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-workload-socket\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.390226 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.390174    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-token\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.390338 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.390259    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-workload-certs\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.390338 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.390287    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-data\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.391365 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.391327    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"]
Sep 19 10:48:39.490779 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.490751    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-workload-certs\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.490867 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.490791    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-podinfo\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.490867 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.490814    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-envoy\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.490867 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.490856    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-workload-socket\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.490980 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.490952    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-envoy\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.491015 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.490987    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-credential-socket\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.491083 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.491011    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istiod-ca-cert\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.491083 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.491064    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p664k\" (UniqueName: \"kubernetes.io/projected/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-kube-api-access-p664k\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.491168 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.491099    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-podinfo\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.491168 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.491129    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb64f\" (UniqueName: \"kubernetes.io/projected/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-kube-api-access-wb64f\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.491168 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.491160    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-workload-socket\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.491303 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.491190    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-data\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.491303 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.491234    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-token\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.491303 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.491260    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-token\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.491303 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.491286    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-workload-certs\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.491510 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.491309    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-data\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.491510 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.491335    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istiod-ca-cert\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.491510 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.491354    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-credential-socket\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.491510 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.491360    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-credential-socket\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.491732 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.491588    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-data\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.491732 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.491715    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-workload-certs\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.491921 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.491896    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istiod-ca-cert\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.492027 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.492001    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-workload-socket\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.493205 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.493186    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-envoy\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.493505 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.493486    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-podinfo\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.498339 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.498318    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb64f\" (UniqueName: \"kubernetes.io/projected/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-kube-api-access-wb64f\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.498442 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.498390    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-token\") pod \"e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") " pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.568385 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.568330    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:39.592816 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.592795    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-podinfo\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.592904 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.592838    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-workload-socket\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.592904 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.592874    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-envoy\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.593013 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.592934    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p664k\" (UniqueName: \"kubernetes.io/projected/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-kube-api-access-p664k\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.593013 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.592997    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-data\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.593133 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.593032    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-token\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.593133 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.593078    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istiod-ca-cert\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.593133 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.593103    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-credential-socket\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.593297 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.593158    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-workload-certs\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.593350 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.593309    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-workload-socket\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.593494 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.593444    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-data\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.593565 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.593504    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-workload-certs\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.593675 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.593648    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-credential-socket\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.593814 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.593791    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istiod-ca-cert\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.595414 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.595396    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-podinfo\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.595690 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.595667    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-envoy\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.601018 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.600985    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-token\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.601138 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.601107    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p664k\" (UniqueName: \"kubernetes.io/projected/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-kube-api-access-p664k\") pod \"e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") " pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.692519 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.692491    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:39.843995 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.843971    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"]
Sep 19 10:48:39.845468 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:48:39.845427    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb368f1b5_0779_4f8f_86ff_d6016e2bdc84.slice/crio-9ba09a78ebf8407066fbce62a4c2abc2cc51a103a5a8245f75eb17aa50695d19 WatchSource:0}: Error finding container 9ba09a78ebf8407066fbce62a4c2abc2cc51a103a5a8245f75eb17aa50695d19: Status 404 returned error can't find the container with id 9ba09a78ebf8407066fbce62a4c2abc2cc51a103a5a8245f75eb17aa50695d19
Sep 19 10:48:39.967241 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:39.967213    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"]
Sep 19 10:48:39.968731 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:48:39.968706    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aa3bf86_e87d_4ba9_b4ce_789c766e2cc9.slice/crio-7c95ece5330dcebc64d550de79403da35d37c58b70fc9eb6baff334321fbc2e5 WatchSource:0}: Error finding container 7c95ece5330dcebc64d550de79403da35d37c58b70fc9eb6baff334321fbc2e5: Status 404 returned error can't find the container with id 7c95ece5330dcebc64d550de79403da35d37c58b70fc9eb6baff334321fbc2e5
Sep 19 10:48:40.812536 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:40.812499    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx" event={"ID":"b368f1b5-0779-4f8f-86ff-d6016e2bdc84","Type":"ContainerStarted","Data":"9ba09a78ebf8407066fbce62a4c2abc2cc51a103a5a8245f75eb17aa50695d19"}
Sep 19 10:48:40.814092 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:40.814056    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p" event={"ID":"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9","Type":"ContainerStarted","Data":"7c95ece5330dcebc64d550de79403da35d37c58b70fc9eb6baff334321fbc2e5"}
Sep 19 10:48:42.975309 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:42.975253    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:48:42.975764 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:42.975360    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:48:42.975764 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:42.975405    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:48:42.979059 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:42.976386    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:48:42.979059 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:42.976462    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:48:42.979059 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:42.976501    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:48:43.833932 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:43.833899    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx" event={"ID":"b368f1b5-0779-4f8f-86ff-d6016e2bdc84","Type":"ContainerStarted","Data":"af2254b9483c8ed0f892fa35eb348de6ede840efa8f7fc5e6190625d4efd7c18"}
Sep 19 10:48:43.835723 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:43.835689    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p" event={"ID":"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9","Type":"ContainerStarted","Data":"34fb6d22a9286df728c08bbd7d5976eec7019b9481b3784a451a6ed7062c9565"}
Sep 19 10:48:43.855172 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:43.855132    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx" podStartSLOduration=1.7278285740000001 podStartE2EDuration="4.855116656s" totalImagesPullingTime="3.127288082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:48:43.852331562 +0000 UTC m=+733.348637654" watchObservedRunningTime="2026-09-19 10:48:43.855116656 +0000 UTC m=+733.351422752"
Sep 19 10:48:43.874582 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:43.874523    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p" podStartSLOduration=1.8692283669999998 podStartE2EDuration="4.874501695s" totalImagesPullingTime="3.005273328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:48:43.871308668 +0000 UTC m=+733.367614761" watchObservedRunningTime="2026-09-19 10:48:43.874501695 +0000 UTC m=+733.370807788"
Sep 19 10:48:43.987242 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:43.987205    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"]
Sep 19 10:48:43.990796 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:43.990776    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"
Sep 19 10:48:43.993269 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:43.993248    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-5116ec-serving-cert\""
Sep 19 10:48:44.001975 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.001952    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"]
Sep 19 10:48:44.040129 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.040101    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/615a8f92-a0a9-4b40-b723-5422c035dda5-metrics-certs\") pod \"maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx\" (UID: \"615a8f92-a0a9-4b40-b723-5422c035dda5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"
Sep 19 10:48:44.040280 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.040173    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tbxs\" (UniqueName: \"kubernetes.io/projected/615a8f92-a0a9-4b40-b723-5422c035dda5-kube-api-access-5tbxs\") pod \"maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx\" (UID: \"615a8f92-a0a9-4b40-b723-5422c035dda5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"
Sep 19 10:48:44.040329 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.040298    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/615a8f92-a0a9-4b40-b723-5422c035dda5-maas-api-tls\") pod \"maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx\" (UID: \"615a8f92-a0a9-4b40-b723-5422c035dda5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"
Sep 19 10:48:44.040381 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.040343    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/615a8f92-a0a9-4b40-b723-5422c035dda5-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx\" (UID: \"615a8f92-a0a9-4b40-b723-5422c035dda5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"
Sep 19 10:48:44.141696 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.141665    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/615a8f92-a0a9-4b40-b723-5422c035dda5-maas-api-tls\") pod \"maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx\" (UID: \"615a8f92-a0a9-4b40-b723-5422c035dda5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"
Sep 19 10:48:44.141877 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.141703    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/615a8f92-a0a9-4b40-b723-5422c035dda5-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx\" (UID: \"615a8f92-a0a9-4b40-b723-5422c035dda5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"
Sep 19 10:48:44.141877 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.141788    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/615a8f92-a0a9-4b40-b723-5422c035dda5-metrics-certs\") pod \"maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx\" (UID: \"615a8f92-a0a9-4b40-b723-5422c035dda5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"
Sep 19 10:48:44.141995 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:48:44.141876    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-serving-cert: secret "maas-api-e2e-worker-w4-5116ec-serving-cert" not found
Sep 19 10:48:44.141995 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:48:44.141944    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/615a8f92-a0a9-4b40-b723-5422c035dda5-maas-api-tls podName:615a8f92-a0a9-4b40-b723-5422c035dda5 nodeName:}" failed. No retries permitted until 2026-09-19 10:48:44.641925957 +0000 UTC m=+734.138232048 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/615a8f92-a0a9-4b40-b723-5422c035dda5-maas-api-tls") pod "maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx" (UID: "615a8f92-a0a9-4b40-b723-5422c035dda5") : secret "maas-api-e2e-worker-w4-5116ec-serving-cert" not found
Sep 19 10:48:44.141995 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.141884    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tbxs\" (UniqueName: \"kubernetes.io/projected/615a8f92-a0a9-4b40-b723-5422c035dda5-kube-api-access-5tbxs\") pod \"maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx\" (UID: \"615a8f92-a0a9-4b40-b723-5422c035dda5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"
Sep 19 10:48:44.142420 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.142400    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/615a8f92-a0a9-4b40-b723-5422c035dda5-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx\" (UID: \"615a8f92-a0a9-4b40-b723-5422c035dda5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"
Sep 19 10:48:44.144452 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.144431    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/615a8f92-a0a9-4b40-b723-5422c035dda5-metrics-certs\") pod \"maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx\" (UID: \"615a8f92-a0a9-4b40-b723-5422c035dda5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"
Sep 19 10:48:44.153429 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.153408    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tbxs\" (UniqueName: \"kubernetes.io/projected/615a8f92-a0a9-4b40-b723-5422c035dda5-kube-api-access-5tbxs\") pod \"maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx\" (UID: \"615a8f92-a0a9-4b40-b723-5422c035dda5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"
Sep 19 10:48:44.569477 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.569397    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:44.574651 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.574626    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:44.647952 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.647923    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/615a8f92-a0a9-4b40-b723-5422c035dda5-maas-api-tls\") pod \"maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx\" (UID: \"615a8f92-a0a9-4b40-b723-5422c035dda5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"
Sep 19 10:48:44.650623 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.650600    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/615a8f92-a0a9-4b40-b723-5422c035dda5-maas-api-tls\") pod \"maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx\" (UID: \"615a8f92-a0a9-4b40-b723-5422c035dda5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"
Sep 19 10:48:44.693490 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.693460    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:44.698230 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.698208    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:44.727062 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.727023    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw"]
Sep 19 10:48:44.730929 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.730914    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw"
Sep 19 10:48:44.733753 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.733734    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-5116ec\""
Sep 19 10:48:44.733843 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.733742    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-5116ec-dockercfg-56fb5\""
Sep 19 10:48:44.743513 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.743491    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw"]
Sep 19 10:48:44.843349 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.843256    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:44.843349 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.843312    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:44.844257 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.844226    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:48:44.844384 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.844319    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:48:44.850672 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.850637    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84c64e61-3aca-46d8-bfd6-065287bf0fb8-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw\" (UID: \"84c64e61-3aca-46d8-bfd6-065287bf0fb8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw"
Sep 19 10:48:44.850782 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.850734    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2d2q\" (UniqueName: \"kubernetes.io/projected/84c64e61-3aca-46d8-bfd6-065287bf0fb8-kube-api-access-m2d2q\") pod \"payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw\" (UID: \"84c64e61-3aca-46d8-bfd6-065287bf0fb8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw"
Sep 19 10:48:44.907066 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.907009    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"
Sep 19 10:48:44.952405 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.951842    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84c64e61-3aca-46d8-bfd6-065287bf0fb8-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw\" (UID: \"84c64e61-3aca-46d8-bfd6-065287bf0fb8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw"
Sep 19 10:48:44.952405 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.952026    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2d2q\" (UniqueName: \"kubernetes.io/projected/84c64e61-3aca-46d8-bfd6-065287bf0fb8-kube-api-access-m2d2q\") pod \"payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw\" (UID: \"84c64e61-3aca-46d8-bfd6-065287bf0fb8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw"
Sep 19 10:48:44.952827 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.952801    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84c64e61-3aca-46d8-bfd6-065287bf0fb8-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw\" (UID: \"84c64e61-3aca-46d8-bfd6-065287bf0fb8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw"
Sep 19 10:48:44.961457 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:44.961431    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2d2q\" (UniqueName: \"kubernetes.io/projected/84c64e61-3aca-46d8-bfd6-065287bf0fb8-kube-api-access-m2d2q\") pod \"payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw\" (UID: \"84c64e61-3aca-46d8-bfd6-065287bf0fb8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw"
Sep 19 10:48:45.041332 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:45.041300    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw"
Sep 19 10:48:45.181940 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:45.181915    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"]
Sep 19 10:48:45.184027 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:48:45.184005    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod615a8f92_a0a9_4b40_b723_5422c035dda5.slice/crio-ccd7b968c9206e69ab5a5d6be8cafa5d4f6501f428878c1c2837013ec73269ac WatchSource:0}: Error finding container ccd7b968c9206e69ab5a5d6be8cafa5d4f6501f428878c1c2837013ec73269ac: Status 404 returned error can't find the container with id ccd7b968c9206e69ab5a5d6be8cafa5d4f6501f428878c1c2837013ec73269ac
Sep 19 10:48:45.318096 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:45.318070    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw"]
Sep 19 10:48:45.319746 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:48:45.319709    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84c64e61_3aca_46d8_bfd6_065287bf0fb8.slice/crio-3f9c057453eeeb7e63b0a241808f967c7d795e7e0c2955a628525d6598f430f4 WatchSource:0}: Error finding container 3f9c057453eeeb7e63b0a241808f967c7d795e7e0c2955a628525d6598f430f4: Status 404 returned error can't find the container with id 3f9c057453eeeb7e63b0a241808f967c7d795e7e0c2955a628525d6598f430f4
Sep 19 10:48:45.852617 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:45.852581    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw" event={"ID":"84c64e61-3aca-46d8-bfd6-065287bf0fb8","Type":"ContainerStarted","Data":"e4f821a4a56b069339bf40001b2df7178a6d97b882c4c3d710bfa7cf8c93b8f6"}
Sep 19 10:48:45.852812 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:45.852648    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw" event={"ID":"84c64e61-3aca-46d8-bfd6-065287bf0fb8","Type":"ContainerStarted","Data":"3f9c057453eeeb7e63b0a241808f967c7d795e7e0c2955a628525d6598f430f4"}
Sep 19 10:48:45.852812 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:45.852799    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw"
Sep 19 10:48:45.854728 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:45.854701    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx" event={"ID":"615a8f92-a0a9-4b40-b723-5422c035dda5","Type":"ContainerStarted","Data":"3d4d13ad37e495e420d436e0cae221fb6ffb9e23031e90a399e6c77084d4c9f6"}
Sep 19 10:48:45.854874 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:45.854748    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx" event={"ID":"615a8f92-a0a9-4b40-b723-5422c035dda5","Type":"ContainerStarted","Data":"ccd7b968c9206e69ab5a5d6be8cafa5d4f6501f428878c1c2837013ec73269ac"}
Sep 19 10:48:45.854874 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:45.854864    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"
Sep 19 10:48:45.872029 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:45.871977    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw" podStartSLOduration=1.871956916 podStartE2EDuration="1.871956916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:48:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:48:45.869607077 +0000 UTC m=+735.365913184" watchObservedRunningTime="2026-09-19 10:48:45.871956916 +0000 UTC m=+735.368263009"
Sep 19 10:48:45.888225 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:45.888177    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx" podStartSLOduration=2.429974548 podStartE2EDuration="2.888162036s" totalImagesPullingTime="458.187488ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:48:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:48:45.887150934 +0000 UTC m=+735.383457037" watchObservedRunningTime="2026-09-19 10:48:45.888162036 +0000 UTC m=+735.384468127"
Sep 19 10:48:48.487113 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.487074    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx"]
Sep 19 10:48:48.491095 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.491075    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx"
Sep 19 10:48:48.493628 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.493606    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-1907c1-dockercfg-h5nv6\""
Sep 19 10:48:48.493767 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.493659    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-1907c1\""
Sep 19 10:48:48.501430 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.501407    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx"]
Sep 19 10:48:48.590203 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.590169    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p955\" (UniqueName: \"kubernetes.io/projected/55dca516-aa13-4415-b070-c3e64fc30de9-kube-api-access-2p955\") pod \"payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx\" (UID: \"55dca516-aa13-4415-b070-c3e64fc30de9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx"
Sep 19 10:48:48.590372 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.590300    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55dca516-aa13-4415-b070-c3e64fc30de9-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx\" (UID: \"55dca516-aa13-4415-b070-c3e64fc30de9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx"
Sep 19 10:48:48.601854 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.601816    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck"]
Sep 19 10:48:48.606297 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.606278    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck"
Sep 19 10:48:48.614531 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.614507    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck"]
Sep 19 10:48:48.691430 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.691400    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rskzn\" (UniqueName: \"kubernetes.io/projected/61fc1267-dfbc-480c-b6df-fdc193061041-kube-api-access-rskzn\") pod \"payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck\" (UID: \"61fc1267-dfbc-480c-b6df-fdc193061041\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck"
Sep 19 10:48:48.691595 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.691451    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p955\" (UniqueName: \"kubernetes.io/projected/55dca516-aa13-4415-b070-c3e64fc30de9-kube-api-access-2p955\") pod \"payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx\" (UID: \"55dca516-aa13-4415-b070-c3e64fc30de9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx"
Sep 19 10:48:48.691595 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.691517    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61fc1267-dfbc-480c-b6df-fdc193061041-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck\" (UID: \"61fc1267-dfbc-480c-b6df-fdc193061041\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck"
Sep 19 10:48:48.691595 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.691548    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55dca516-aa13-4415-b070-c3e64fc30de9-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx\" (UID: \"55dca516-aa13-4415-b070-c3e64fc30de9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx"
Sep 19 10:48:48.692192 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.692171    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55dca516-aa13-4415-b070-c3e64fc30de9-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx\" (UID: \"55dca516-aa13-4415-b070-c3e64fc30de9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx"
Sep 19 10:48:48.699759 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.699731    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p955\" (UniqueName: \"kubernetes.io/projected/55dca516-aa13-4415-b070-c3e64fc30de9-kube-api-access-2p955\") pod \"payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx\" (UID: \"55dca516-aa13-4415-b070-c3e64fc30de9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx"
Sep 19 10:48:48.792134 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.792023    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rskzn\" (UniqueName: \"kubernetes.io/projected/61fc1267-dfbc-480c-b6df-fdc193061041-kube-api-access-rskzn\") pod \"payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck\" (UID: \"61fc1267-dfbc-480c-b6df-fdc193061041\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck"
Sep 19 10:48:48.792270 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.792151    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61fc1267-dfbc-480c-b6df-fdc193061041-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck\" (UID: \"61fc1267-dfbc-480c-b6df-fdc193061041\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck"
Sep 19 10:48:48.792817 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.792799    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61fc1267-dfbc-480c-b6df-fdc193061041-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck\" (UID: \"61fc1267-dfbc-480c-b6df-fdc193061041\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck"
Sep 19 10:48:48.799851 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.799823    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rskzn\" (UniqueName: \"kubernetes.io/projected/61fc1267-dfbc-480c-b6df-fdc193061041-kube-api-access-rskzn\") pod \"payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck\" (UID: \"61fc1267-dfbc-480c-b6df-fdc193061041\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck"
Sep 19 10:48:48.802554 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.802530    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx"
Sep 19 10:48:48.917841 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:48.917816    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck"
Sep 19 10:48:49.075972 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.075949    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx"]
Sep 19 10:48:49.077736 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:48:49.077711    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55dca516_aa13_4415_b070_c3e64fc30de9.slice/crio-e7bbf396242ca0a98cf157f0898f200aa2f27de9e4fcea3d6d88b27221c0b4d4 WatchSource:0}: Error finding container e7bbf396242ca0a98cf157f0898f200aa2f27de9e4fcea3d6d88b27221c0b4d4: Status 404 returned error can't find the container with id e7bbf396242ca0a98cf157f0898f200aa2f27de9e4fcea3d6d88b27221c0b4d4
Sep 19 10:48:49.093910 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.093882    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"]
Sep 19 10:48:49.099207 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.099185    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"
Sep 19 10:48:49.102112 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.101566    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-6a0d36-serving-cert\""
Sep 19 10:48:49.108275 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.108255    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"]
Sep 19 10:48:49.196573 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.196547    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck"]
Sep 19 10:48:49.196890 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.196873    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c1b3443-b58f-4637-a0fb-53b219d92689-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw\" (UID: \"3c1b3443-b58f-4637-a0fb-53b219d92689\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"
Sep 19 10:48:49.196973 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.196958    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwfmx\" (UniqueName: \"kubernetes.io/projected/3c1b3443-b58f-4637-a0fb-53b219d92689-kube-api-access-cwfmx\") pod \"maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw\" (UID: \"3c1b3443-b58f-4637-a0fb-53b219d92689\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"
Sep 19 10:48:49.197013 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.196987    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c1b3443-b58f-4637-a0fb-53b219d92689-maas-api-tls\") pod \"maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw\" (UID: \"3c1b3443-b58f-4637-a0fb-53b219d92689\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"
Sep 19 10:48:49.197013 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.197005    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c1b3443-b58f-4637-a0fb-53b219d92689-metrics-certs\") pod \"maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw\" (UID: \"3c1b3443-b58f-4637-a0fb-53b219d92689\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"
Sep 19 10:48:49.198581 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:48:49.198556    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61fc1267_dfbc_480c_b6df_fdc193061041.slice/crio-8b26cb79381f74037c27ebd7f43fa15256848cb0e4154224f28c5359eef942d8 WatchSource:0}: Error finding container 8b26cb79381f74037c27ebd7f43fa15256848cb0e4154224f28c5359eef942d8: Status 404 returned error can't find the container with id 8b26cb79381f74037c27ebd7f43fa15256848cb0e4154224f28c5359eef942d8
Sep 19 10:48:49.298390 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.298360    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c1b3443-b58f-4637-a0fb-53b219d92689-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw\" (UID: \"3c1b3443-b58f-4637-a0fb-53b219d92689\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"
Sep 19 10:48:49.298507 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.298434    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwfmx\" (UniqueName: \"kubernetes.io/projected/3c1b3443-b58f-4637-a0fb-53b219d92689-kube-api-access-cwfmx\") pod \"maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw\" (UID: \"3c1b3443-b58f-4637-a0fb-53b219d92689\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"
Sep 19 10:48:49.298507 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.298464    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c1b3443-b58f-4637-a0fb-53b219d92689-maas-api-tls\") pod \"maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw\" (UID: \"3c1b3443-b58f-4637-a0fb-53b219d92689\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"
Sep 19 10:48:49.298507 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.298485    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c1b3443-b58f-4637-a0fb-53b219d92689-metrics-certs\") pod \"maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw\" (UID: \"3c1b3443-b58f-4637-a0fb-53b219d92689\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"
Sep 19 10:48:49.298636 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:48:49.298611    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-serving-cert: secret "maas-api-e2e-inf-a-6a0d36-serving-cert" not found
Sep 19 10:48:49.298713 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:48:49.298690    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c1b3443-b58f-4637-a0fb-53b219d92689-maas-api-tls podName:3c1b3443-b58f-4637-a0fb-53b219d92689 nodeName:}" failed. No retries permitted until 2026-09-19 10:48:49.798651008 +0000 UTC m=+739.294957081 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3c1b3443-b58f-4637-a0fb-53b219d92689-maas-api-tls") pod "maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw" (UID: "3c1b3443-b58f-4637-a0fb-53b219d92689") : secret "maas-api-e2e-inf-a-6a0d36-serving-cert" not found
Sep 19 10:48:49.299155 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.299137    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c1b3443-b58f-4637-a0fb-53b219d92689-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw\" (UID: \"3c1b3443-b58f-4637-a0fb-53b219d92689\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"
Sep 19 10:48:49.300806 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.300788    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c1b3443-b58f-4637-a0fb-53b219d92689-metrics-certs\") pod \"maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw\" (UID: \"3c1b3443-b58f-4637-a0fb-53b219d92689\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"
Sep 19 10:48:49.310952 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.310915    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwfmx\" (UniqueName: \"kubernetes.io/projected/3c1b3443-b58f-4637-a0fb-53b219d92689-kube-api-access-cwfmx\") pod \"maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw\" (UID: \"3c1b3443-b58f-4637-a0fb-53b219d92689\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"
Sep 19 10:48:49.802330 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.802187    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c1b3443-b58f-4637-a0fb-53b219d92689-maas-api-tls\") pod \"maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw\" (UID: \"3c1b3443-b58f-4637-a0fb-53b219d92689\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"
Sep 19 10:48:49.804990 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.804961    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c1b3443-b58f-4637-a0fb-53b219d92689-maas-api-tls\") pod \"maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw\" (UID: \"3c1b3443-b58f-4637-a0fb-53b219d92689\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"
Sep 19 10:48:49.885366 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.885328    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck" event={"ID":"61fc1267-dfbc-480c-b6df-fdc193061041","Type":"ContainerStarted","Data":"42f892eeb8906de49d80ac4ecf44e5ced8d147432b4691cd6b36d6fbb141a7d5"}
Sep 19 10:48:49.885508 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.885391    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck"
Sep 19 10:48:49.885508 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.885412    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck" event={"ID":"61fc1267-dfbc-480c-b6df-fdc193061041","Type":"ContainerStarted","Data":"8b26cb79381f74037c27ebd7f43fa15256848cb0e4154224f28c5359eef942d8"}
Sep 19 10:48:49.887015 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.886991    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx" event={"ID":"55dca516-aa13-4415-b070-c3e64fc30de9","Type":"ContainerStarted","Data":"51e03c08e6288f9164ab8f4f9cfe0fe6f78d2fba4359738ba1a4b0e80f7748ff"}
Sep 19 10:48:49.887165 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.887028    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx" event={"ID":"55dca516-aa13-4415-b070-c3e64fc30de9","Type":"ContainerStarted","Data":"e7bbf396242ca0a98cf157f0898f200aa2f27de9e4fcea3d6d88b27221c0b4d4"}
Sep 19 10:48:49.887165 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.887071    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx"
Sep 19 10:48:49.937906 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.937864    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck" podStartSLOduration=1.9378508 podStartE2EDuration="1.9378508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:48:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:48:49.936528529 +0000 UTC m=+739.432834622" watchObservedRunningTime="2026-09-19 10:48:49.9378508 +0000 UTC m=+739.434156891"
Sep 19 10:48:49.964378 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:49.964337    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx" podStartSLOduration=1.9643243620000002 podStartE2EDuration="1.964324362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:48:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:48:49.9620929 +0000 UTC m=+739.458398995" watchObservedRunningTime="2026-09-19 10:48:49.964324362 +0000 UTC m=+739.460630483"
Sep 19 10:48:50.020881 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:50.020856    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"
Sep 19 10:48:50.312652 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:50.312606    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"]
Sep 19 10:48:50.413566 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:50.413526    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"]
Sep 19 10:48:50.419273 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:50.418847    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"
Sep 19 10:48:50.421677 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:50.421651    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-7bcaa9-serving-cert\""
Sep 19 10:48:50.428065 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:50.428010    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"]
Sep 19 10:48:50.511007 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:50.510970    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl2ts\" (UniqueName: \"kubernetes.io/projected/0b6dc030-a71c-465c-8788-349970253cb6-kube-api-access-hl2ts\") pod \"maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz\" (UID: \"0b6dc030-a71c-465c-8788-349970253cb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"
Sep 19 10:48:50.511193 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:50.511017    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b6dc030-a71c-465c-8788-349970253cb6-maas-api-tls\") pod \"maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz\" (UID: \"0b6dc030-a71c-465c-8788-349970253cb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"
Sep 19 10:48:50.511193 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:50.511107    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b6dc030-a71c-465c-8788-349970253cb6-metrics-certs\") pod \"maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz\" (UID: \"0b6dc030-a71c-465c-8788-349970253cb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"
Sep 19 10:48:50.511193 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:50.511148    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b6dc030-a71c-465c-8788-349970253cb6-openshift-service-ca\") pod \"maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz\" (UID: \"0b6dc030-a71c-465c-8788-349970253cb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"
Sep 19 10:48:50.612034 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:50.611992    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b6dc030-a71c-465c-8788-349970253cb6-maas-api-tls\") pod \"maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz\" (UID: \"0b6dc030-a71c-465c-8788-349970253cb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"
Sep 19 10:48:50.612233 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:50.612111    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b6dc030-a71c-465c-8788-349970253cb6-metrics-certs\") pod \"maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz\" (UID: \"0b6dc030-a71c-465c-8788-349970253cb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"
Sep 19 10:48:50.612233 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:50.612174    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b6dc030-a71c-465c-8788-349970253cb6-openshift-service-ca\") pod \"maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz\" (UID: \"0b6dc030-a71c-465c-8788-349970253cb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"
Sep 19 10:48:50.612233 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:48:50.612181    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-serving-cert: secret "maas-api-e2e-autores-7bcaa9-serving-cert" not found
Sep 19 10:48:50.612402 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:48:50.612274    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b6dc030-a71c-465c-8788-349970253cb6-maas-api-tls podName:0b6dc030-a71c-465c-8788-349970253cb6 nodeName:}" failed. No retries permitted until 2026-09-19 10:48:51.112237629 +0000 UTC m=+740.608543705 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0b6dc030-a71c-465c-8788-349970253cb6-maas-api-tls") pod "maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz" (UID: "0b6dc030-a71c-465c-8788-349970253cb6") : secret "maas-api-e2e-autores-7bcaa9-serving-cert" not found
Sep 19 10:48:50.612402 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:50.612366    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl2ts\" (UniqueName: \"kubernetes.io/projected/0b6dc030-a71c-465c-8788-349970253cb6-kube-api-access-hl2ts\") pod \"maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz\" (UID: \"0b6dc030-a71c-465c-8788-349970253cb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"
Sep 19 10:48:50.613036 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:50.613013    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b6dc030-a71c-465c-8788-349970253cb6-openshift-service-ca\") pod \"maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz\" (UID: \"0b6dc030-a71c-465c-8788-349970253cb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"
Sep 19 10:48:50.614825 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:50.614800    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b6dc030-a71c-465c-8788-349970253cb6-metrics-certs\") pod \"maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz\" (UID: \"0b6dc030-a71c-465c-8788-349970253cb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"
Sep 19 10:48:50.623491 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:50.623464    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl2ts\" (UniqueName: \"kubernetes.io/projected/0b6dc030-a71c-465c-8788-349970253cb6-kube-api-access-hl2ts\") pod \"maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz\" (UID: \"0b6dc030-a71c-465c-8788-349970253cb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"
Sep 19 10:48:50.893827 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:50.893794    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw" event={"ID":"3c1b3443-b58f-4637-a0fb-53b219d92689","Type":"ContainerStarted","Data":"25db93a258988bccbfa39763c587f6f63c6f00d5312eb5e358068a9c7273c1d5"}
Sep 19 10:48:51.119875 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.119842    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b6dc030-a71c-465c-8788-349970253cb6-maas-api-tls\") pod \"maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz\" (UID: \"0b6dc030-a71c-465c-8788-349970253cb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"
Sep 19 10:48:51.122435 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.122373    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b6dc030-a71c-465c-8788-349970253cb6-maas-api-tls\") pod \"maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz\" (UID: \"0b6dc030-a71c-465c-8788-349970253cb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"
Sep 19 10:48:51.137365 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.137339    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c"]
Sep 19 10:48:51.141512 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.141491    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c"
Sep 19 10:48:51.143991 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.143974    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-7bcaa9\""
Sep 19 10:48:51.151270 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.151144    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c"]
Sep 19 10:48:51.220804 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.220771    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm8r9\" (UniqueName: \"kubernetes.io/projected/02d306ea-2c5b-41ae-bca9-dfacc4ea49e5-kube-api-access-hm8r9\") pod \"payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c\" (UID: \"02d306ea-2c5b-41ae-bca9-dfacc4ea49e5\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c"
Sep 19 10:48:51.220944 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.220824    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02d306ea-2c5b-41ae-bca9-dfacc4ea49e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c\" (UID: \"02d306ea-2c5b-41ae-bca9-dfacc4ea49e5\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c"
Sep 19 10:48:51.321685 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.321650    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm8r9\" (UniqueName: \"kubernetes.io/projected/02d306ea-2c5b-41ae-bca9-dfacc4ea49e5-kube-api-access-hm8r9\") pod \"payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c\" (UID: \"02d306ea-2c5b-41ae-bca9-dfacc4ea49e5\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c"
Sep 19 10:48:51.321895 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.321723    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02d306ea-2c5b-41ae-bca9-dfacc4ea49e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c\" (UID: \"02d306ea-2c5b-41ae-bca9-dfacc4ea49e5\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c"
Sep 19 10:48:51.322498 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.322462    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02d306ea-2c5b-41ae-bca9-dfacc4ea49e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c\" (UID: \"02d306ea-2c5b-41ae-bca9-dfacc4ea49e5\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c"
Sep 19 10:48:51.329351 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.329321    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm8r9\" (UniqueName: \"kubernetes.io/projected/02d306ea-2c5b-41ae-bca9-dfacc4ea49e5-kube-api-access-hm8r9\") pod \"payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c\" (UID: \"02d306ea-2c5b-41ae-bca9-dfacc4ea49e5\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c"
Sep 19 10:48:51.335201 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.335178    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"
Sep 19 10:48:51.457210 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.457173    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c"
Sep 19 10:48:51.614657 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.614626    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"]
Sep 19 10:48:51.617715 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:48:51.617687    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b6dc030_a71c_465c_8788_349970253cb6.slice/crio-2d372b692767eace235abb0b52eb8b2bb0bc886cdff6a4fcaf5243bda0f00b39 WatchSource:0}: Error finding container 2d372b692767eace235abb0b52eb8b2bb0bc886cdff6a4fcaf5243bda0f00b39: Status 404 returned error can't find the container with id 2d372b692767eace235abb0b52eb8b2bb0bc886cdff6a4fcaf5243bda0f00b39
Sep 19 10:48:51.743260 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.743231    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c"]
Sep 19 10:48:51.745426 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:48:51.745400    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02d306ea_2c5b_41ae_bca9_dfacc4ea49e5.slice/crio-7c5da31b4c98f4992f93c56ee94ebebaacb01971954f212ddd5471be8fcd7e3f WatchSource:0}: Error finding container 7c5da31b4c98f4992f93c56ee94ebebaacb01971954f212ddd5471be8fcd7e3f: Status 404 returned error can't find the container with id 7c5da31b4c98f4992f93c56ee94ebebaacb01971954f212ddd5471be8fcd7e3f
Sep 19 10:48:51.874782 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.874747    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"
Sep 19 10:48:51.901912 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.901881    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw" event={"ID":"3c1b3443-b58f-4637-a0fb-53b219d92689","Type":"ContainerStarted","Data":"60e777c1ef1e2db3e073a0e166e7aed2b0279499faa086bca876c901cd12e5fa"}
Sep 19 10:48:51.902312 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.901961    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"
Sep 19 10:48:51.904249 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.904141    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c" event={"ID":"02d306ea-2c5b-41ae-bca9-dfacc4ea49e5","Type":"ContainerStarted","Data":"5a98f1f2842408481b83eb9562a0bc0e8179b5a4ee20e16890b56979cf7ff812"}
Sep 19 10:48:51.904249 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.904191    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c" event={"ID":"02d306ea-2c5b-41ae-bca9-dfacc4ea49e5","Type":"ContainerStarted","Data":"7c5da31b4c98f4992f93c56ee94ebebaacb01971954f212ddd5471be8fcd7e3f"}
Sep 19 10:48:51.904249 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.904220    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c"
Sep 19 10:48:51.905842 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.905811    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz" event={"ID":"0b6dc030-a71c-465c-8788-349970253cb6","Type":"ContainerStarted","Data":"2d372b692767eace235abb0b52eb8b2bb0bc886cdff6a4fcaf5243bda0f00b39"}
Sep 19 10:48:51.921681 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.921631    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw" podStartSLOduration=2.365361545 podStartE2EDuration="2.921617045s" totalImagesPullingTime="556.2555ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:48:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:48:51.919712199 +0000 UTC m=+741.416018292" watchObservedRunningTime="2026-09-19 10:48:51.921617045 +0000 UTC m=+741.417923137"
Sep 19 10:48:51.937986 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:51.937935    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c" podStartSLOduration=0.937917194 podStartE2EDuration="937.917194ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:48:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:48:51.934484136 +0000 UTC m=+741.430790229" watchObservedRunningTime="2026-09-19 10:48:51.937917194 +0000 UTC m=+741.434223288"
Sep 19 10:48:52.913375 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:52.913332    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz" event={"ID":"0b6dc030-a71c-465c-8788-349970253cb6","Type":"ContainerStarted","Data":"7c5cd883a946f2ad9e363bbb8266dbfef73affde7229c255eab5462d5a11cc81"}
Sep 19 10:48:52.913748 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:52.913720    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"
Sep 19 10:48:52.930709 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:52.930663    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz" podStartSLOduration=2.655070958 podStartE2EDuration="2.93064684s" totalImagesPullingTime="275.575882ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:48:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:48:52.928531722 +0000 UTC m=+742.424837813" watchObservedRunningTime="2026-09-19 10:48:52.93064684 +0000 UTC m=+742.426952932"
Sep 19 10:48:56.870169 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:56.870138    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw"
Sep 19 10:48:57.918652 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:57.918625    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"
Sep 19 10:48:58.590070 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:58.590024    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54"]
Sep 19 10:48:58.594036 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:58.594016    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54"
Sep 19 10:48:58.596967 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:58.596949    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-c152d119\""
Sep 19 10:48:58.597096 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:58.597006    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-c152d119-dockercfg-pzmxt\""
Sep 19 10:48:58.619775 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:58.619747    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54"]
Sep 19 10:48:58.702656 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:58.702308    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fbfa74b-166c-49f0-85d9-08a0ab7baf01-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54\" (UID: \"1fbfa74b-166c-49f0-85d9-08a0ab7baf01\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54"
Sep 19 10:48:58.702656 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:58.702364    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnfqs\" (UniqueName: \"kubernetes.io/projected/1fbfa74b-166c-49f0-85d9-08a0ab7baf01-kube-api-access-qnfqs\") pod \"payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54\" (UID: \"1fbfa74b-166c-49f0-85d9-08a0ab7baf01\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54"
Sep 19 10:48:58.803573 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:58.803531    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fbfa74b-166c-49f0-85d9-08a0ab7baf01-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54\" (UID: \"1fbfa74b-166c-49f0-85d9-08a0ab7baf01\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54"
Sep 19 10:48:58.803764 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:58.803593    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnfqs\" (UniqueName: \"kubernetes.io/projected/1fbfa74b-166c-49f0-85d9-08a0ab7baf01-kube-api-access-qnfqs\") pod \"payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54\" (UID: \"1fbfa74b-166c-49f0-85d9-08a0ab7baf01\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54"
Sep 19 10:48:58.804385 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:58.804363    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fbfa74b-166c-49f0-85d9-08a0ab7baf01-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54\" (UID: \"1fbfa74b-166c-49f0-85d9-08a0ab7baf01\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54"
Sep 19 10:48:58.814226 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:58.814199    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnfqs\" (UniqueName: \"kubernetes.io/projected/1fbfa74b-166c-49f0-85d9-08a0ab7baf01-kube-api-access-qnfqs\") pod \"payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54\" (UID: \"1fbfa74b-166c-49f0-85d9-08a0ab7baf01\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54"
Sep 19 10:48:58.905512 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:58.905431    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54"
Sep 19 10:48:58.925467 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:58.925444    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"
Sep 19 10:48:59.202135 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:59.202106    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54"]
Sep 19 10:48:59.204324 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:48:59.204296    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fbfa74b_166c_49f0_85d9_08a0ab7baf01.slice/crio-aef9b4476772fcd049f412c261601eeb2c9ccbea7ebaf10c0a7c1f24e5935ee8 WatchSource:0}: Error finding container aef9b4476772fcd049f412c261601eeb2c9ccbea7ebaf10c0a7c1f24e5935ee8: Status 404 returned error can't find the container with id aef9b4476772fcd049f412c261601eeb2c9ccbea7ebaf10c0a7c1f24e5935ee8
Sep 19 10:48:59.992439 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:59.992338    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54" event={"ID":"1fbfa74b-166c-49f0-85d9-08a0ab7baf01","Type":"ContainerStarted","Data":"a4e56f89ec28dee3d8f27645ebce40f1e60e6883402b59c20a7c69e972a7af6d"}
Sep 19 10:48:59.992439 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:59.992402    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54" event={"ID":"1fbfa74b-166c-49f0-85d9-08a0ab7baf01","Type":"ContainerStarted","Data":"aef9b4476772fcd049f412c261601eeb2c9ccbea7ebaf10c0a7c1f24e5935ee8"}
Sep 19 10:48:59.993137 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:48:59.993069    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54"
Sep 19 10:49:00.011019 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:00.010342    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54" podStartSLOduration=2.010308878 podStartE2EDuration="2.010308878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:48:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:49:00.010091244 +0000 UTC m=+749.506397337" watchObservedRunningTime="2026-09-19 10:49:00.010308878 +0000 UTC m=+749.506614972"
Sep 19 10:49:00.885201 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:00.885165    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 19 10:49:00.885369 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:00.885246    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx" podUID="b368f1b5-0779-4f8f-86ff-d6016e2bdc84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 19 10:49:00.895609 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:00.895570    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck"
Sep 19 10:49:00.895866 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:00.895846    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx"
Sep 19 10:49:02.915908 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:02.915875    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c"
Sep 19 10:49:04.016988 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:04.016956    2576 generic.go:356] "Generic (PLEG): container finished" podID="b368f1b5-0779-4f8f-86ff-d6016e2bdc84" containerID="af2254b9483c8ed0f892fa35eb348de6ede840efa8f7fc5e6190625d4efd7c18" exitCode=0
Sep 19 10:49:04.017463 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:04.017029    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx" event={"ID":"b368f1b5-0779-4f8f-86ff-d6016e2bdc84","Type":"ContainerDied","Data":"af2254b9483c8ed0f892fa35eb348de6ede840efa8f7fc5e6190625d4efd7c18"}
Sep 19 10:49:04.017463 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:04.017449    2576 scope.go:117] "RemoveContainer" containerID="af2254b9483c8ed0f892fa35eb348de6ede840efa8f7fc5e6190625d4efd7c18"
Sep 19 10:49:04.018617 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:04.018589    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:49:04.019943 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:04.019211    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:49:04.019943 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:04.019330    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:49:05.023543 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:05.023505    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx" event={"ID":"b368f1b5-0779-4f8f-86ff-d6016e2bdc84","Type":"ContainerStarted","Data":"fb56bb0a8579dba8d3fa78db3c86fa3c8a336f5edf5911b401d7327183d0b3b9"}
Sep 19 10:49:05.568526 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:05.568490    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:49:05.568735 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:05.568714    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 19 10:49:05.568796 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:05.568764    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx" podUID="b368f1b5-0779-4f8f-86ff-d6016e2bdc84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 19 10:49:06.569277 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:06.569236    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 19 10:49:06.569738 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:06.569307    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx" podUID="b368f1b5-0779-4f8f-86ff-d6016e2bdc84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 19 10:49:07.568809 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:07.568766    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 19 10:49:07.569005 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:07.568836    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx" podUID="b368f1b5-0779-4f8f-86ff-d6016e2bdc84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 19 10:49:08.568909 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:08.568877    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 19 10:49:08.569317 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:08.568943    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx" podUID="b368f1b5-0779-4f8f-86ff-d6016e2bdc84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 19 10:49:09.568859 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:09.568828    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 19 10:49:09.569147 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:09.568894    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx" podUID="b368f1b5-0779-4f8f-86ff-d6016e2bdc84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 19 10:49:10.065926 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:10.065889    2576 generic.go:356] "Generic (PLEG): container finished" podID="1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" containerID="34fb6d22a9286df728c08bbd7d5976eec7019b9481b3784a451a6ed7062c9565" exitCode=0
Sep 19 10:49:10.066142 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:10.065975    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p" event={"ID":"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9","Type":"ContainerDied","Data":"34fb6d22a9286df728c08bbd7d5976eec7019b9481b3784a451a6ed7062c9565"}
Sep 19 10:49:10.066543 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:10.066524    2576 scope.go:117] "RemoveContainer" containerID="34fb6d22a9286df728c08bbd7d5976eec7019b9481b3784a451a6ed7062c9565"
Sep 19 10:49:10.067791 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:10.067760    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:49:10.067892 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:10.067829    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:49:10.067892 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:10.067861    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:49:10.572440 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:10.572407    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:49:10.572908 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:10.572818    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:49:10.573614 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:10.573594    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 10:49:11.073326 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:11.073294    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p" event={"ID":"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9","Type":"ContainerStarted","Data":"911b32135f2bf65ad46996e7745f08d64a9b1d44951577973fb6429aff74ba35"}
Sep 19 10:49:11.693399 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:11.693368    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:49:11.694707 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:11.694681    2576 patch_prober.go:28] interesting pod/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 19 10:49:11.694786 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:11.694741    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p" podUID="1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 19 10:49:12.007070 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:12.006964    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54"
Sep 19 10:49:12.693577 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:12.693537    2576 patch_prober.go:28] interesting pod/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 19 10:49:12.693962 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:12.693599    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p" podUID="1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 19 10:49:13.693407 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:13.693368    2576 patch_prober.go:28] interesting pod/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 19 10:49:13.693636 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:13.693425    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p" podUID="1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 19 10:49:14.693402 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:14.693364    2576 patch_prober.go:28] interesting pod/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 19 10:49:14.693582 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:14.693420    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p" podUID="1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 19 10:49:14.847066 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:14.844140    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:49:15.693087 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:15.693026    2576 patch_prober.go:28] interesting pod/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 19 10:49:15.693259 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:15.693112    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p" podUID="1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 19 10:49:16.697470 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:16.697439    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:49:16.698554 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:16.698537    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:49:20.301724 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:20.301693    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54"]
Sep 19 10:49:20.302137 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:20.301916    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54" podUID="1fbfa74b-166c-49f0-85d9-08a0ab7baf01" containerName="payload-pre-processing" containerID="cri-o://a4e56f89ec28dee3d8f27645ebce40f1e60e6883402b59c20a7c69e972a7af6d" gracePeriod=30
Sep 19 10:49:20.562352 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:20.562285    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54"
Sep 19 10:49:20.660591 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:20.660563    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnfqs\" (UniqueName: \"kubernetes.io/projected/1fbfa74b-166c-49f0-85d9-08a0ab7baf01-kube-api-access-qnfqs\") pod \"1fbfa74b-166c-49f0-85d9-08a0ab7baf01\" (UID: \"1fbfa74b-166c-49f0-85d9-08a0ab7baf01\") "
Sep 19 10:49:20.660762 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:20.660613    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fbfa74b-166c-49f0-85d9-08a0ab7baf01-plugins-config-volume\") pod \"1fbfa74b-166c-49f0-85d9-08a0ab7baf01\" (UID: \"1fbfa74b-166c-49f0-85d9-08a0ab7baf01\") "
Sep 19 10:49:20.661019 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:20.660994    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fbfa74b-166c-49f0-85d9-08a0ab7baf01-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1fbfa74b-166c-49f0-85d9-08a0ab7baf01" (UID: "1fbfa74b-166c-49f0-85d9-08a0ab7baf01"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:49:20.663152 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:20.663121    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fbfa74b-166c-49f0-85d9-08a0ab7baf01-kube-api-access-qnfqs" (OuterVolumeSpecName: "kube-api-access-qnfqs") pod "1fbfa74b-166c-49f0-85d9-08a0ab7baf01" (UID: "1fbfa74b-166c-49f0-85d9-08a0ab7baf01"). InnerVolumeSpecName "kube-api-access-qnfqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:49:20.761628 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:20.761596    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnfqs\" (UniqueName: \"kubernetes.io/projected/1fbfa74b-166c-49f0-85d9-08a0ab7baf01-kube-api-access-qnfqs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:49:20.761753 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:20.761634    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fbfa74b-166c-49f0-85d9-08a0ab7baf01-plugins-config-volume\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:49:21.136822 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:21.136785    2576 generic.go:356] "Generic (PLEG): container finished" podID="1fbfa74b-166c-49f0-85d9-08a0ab7baf01" containerID="a4e56f89ec28dee3d8f27645ebce40f1e60e6883402b59c20a7c69e972a7af6d" exitCode=0
Sep 19 10:49:21.137008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:21.136847    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54"
Sep 19 10:49:21.137008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:21.136876    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54" event={"ID":"1fbfa74b-166c-49f0-85d9-08a0ab7baf01","Type":"ContainerDied","Data":"a4e56f89ec28dee3d8f27645ebce40f1e60e6883402b59c20a7c69e972a7af6d"}
Sep 19 10:49:21.137008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:21.136929    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54" event={"ID":"1fbfa74b-166c-49f0-85d9-08a0ab7baf01","Type":"ContainerDied","Data":"aef9b4476772fcd049f412c261601eeb2c9ccbea7ebaf10c0a7c1f24e5935ee8"}
Sep 19 10:49:21.137008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:21.136951    2576 scope.go:117] "RemoveContainer" containerID="a4e56f89ec28dee3d8f27645ebce40f1e60e6883402b59c20a7c69e972a7af6d"
Sep 19 10:49:21.146960 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:21.146940    2576 scope.go:117] "RemoveContainer" containerID="a4e56f89ec28dee3d8f27645ebce40f1e60e6883402b59c20a7c69e972a7af6d"
Sep 19 10:49:21.147236 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:49:21.147209    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4e56f89ec28dee3d8f27645ebce40f1e60e6883402b59c20a7c69e972a7af6d\": container with ID starting with a4e56f89ec28dee3d8f27645ebce40f1e60e6883402b59c20a7c69e972a7af6d not found: ID does not exist" containerID="a4e56f89ec28dee3d8f27645ebce40f1e60e6883402b59c20a7c69e972a7af6d"
Sep 19 10:49:21.147334 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:21.147246    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4e56f89ec28dee3d8f27645ebce40f1e60e6883402b59c20a7c69e972a7af6d"} err="failed to get container status \"a4e56f89ec28dee3d8f27645ebce40f1e60e6883402b59c20a7c69e972a7af6d\": rpc error: code = NotFound desc = could not find container \"a4e56f89ec28dee3d8f27645ebce40f1e60e6883402b59c20a7c69e972a7af6d\": container with ID starting with a4e56f89ec28dee3d8f27645ebce40f1e60e6883402b59c20a7c69e972a7af6d not found: ID does not exist"
Sep 19 10:49:21.153683 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:21.153658    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54"]
Sep 19 10:49:21.157921 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:21.157905    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c152d119-56bd48b4b-hvq54"]
Sep 19 10:49:23.002277 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:49:23.002238    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fbfa74b-166c-49f0-85d9-08a0ab7baf01" path="/var/lib/kubelet/pods/1fbfa74b-166c-49f0-85d9-08a0ab7baf01/volumes"
Sep 19 10:50:00.627225 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.627140    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"]
Sep 19 10:50:00.627753 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.627593    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fbfa74b-166c-49f0-85d9-08a0ab7baf01" containerName="payload-pre-processing"
Sep 19 10:50:00.627753 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.627605    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fbfa74b-166c-49f0-85d9-08a0ab7baf01" containerName="payload-pre-processing"
Sep 19 10:50:00.627753 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.627695    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fbfa74b-166c-49f0-85d9-08a0ab7baf01" containerName="payload-pre-processing"
Sep 19 10:50:00.631016 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.630987    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"
Sep 19 10:50:00.633669 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.633632    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d303059f-serving-cert\""
Sep 19 10:50:00.642097 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.642074    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"]
Sep 19 10:50:00.736848 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.736816    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mgqm\" (UniqueName: \"kubernetes.io/projected/3750a5b5-5b59-4bcf-b9f5-d07434945f20-kube-api-access-7mgqm\") pod \"maas-api-e2e-ait-d303059f-6f5fc74575-kll96\" (UID: \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"
Sep 19 10:50:00.736983 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.736880    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3750a5b5-5b59-4bcf-b9f5-d07434945f20-maas-api-tls\") pod \"maas-api-e2e-ait-d303059f-6f5fc74575-kll96\" (UID: \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"
Sep 19 10:50:00.737136 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.737015    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3750a5b5-5b59-4bcf-b9f5-d07434945f20-metrics-certs\") pod \"maas-api-e2e-ait-d303059f-6f5fc74575-kll96\" (UID: \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"
Sep 19 10:50:00.737136 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.737067    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3750a5b5-5b59-4bcf-b9f5-d07434945f20-openshift-service-ca\") pod \"maas-api-e2e-ait-d303059f-6f5fc74575-kll96\" (UID: \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"
Sep 19 10:50:00.838024 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.837998    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mgqm\" (UniqueName: \"kubernetes.io/projected/3750a5b5-5b59-4bcf-b9f5-d07434945f20-kube-api-access-7mgqm\") pod \"maas-api-e2e-ait-d303059f-6f5fc74575-kll96\" (UID: \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"
Sep 19 10:50:00.838137 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.838090    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3750a5b5-5b59-4bcf-b9f5-d07434945f20-maas-api-tls\") pod \"maas-api-e2e-ait-d303059f-6f5fc74575-kll96\" (UID: \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"
Sep 19 10:50:00.838193 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.838174    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3750a5b5-5b59-4bcf-b9f5-d07434945f20-metrics-certs\") pod \"maas-api-e2e-ait-d303059f-6f5fc74575-kll96\" (UID: \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"
Sep 19 10:50:00.838236 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.838195    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3750a5b5-5b59-4bcf-b9f5-d07434945f20-openshift-service-ca\") pod \"maas-api-e2e-ait-d303059f-6f5fc74575-kll96\" (UID: \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"
Sep 19 10:50:00.838851 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.838833    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3750a5b5-5b59-4bcf-b9f5-d07434945f20-openshift-service-ca\") pod \"maas-api-e2e-ait-d303059f-6f5fc74575-kll96\" (UID: \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"
Sep 19 10:50:00.840569 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.840545    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3750a5b5-5b59-4bcf-b9f5-d07434945f20-maas-api-tls\") pod \"maas-api-e2e-ait-d303059f-6f5fc74575-kll96\" (UID: \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"
Sep 19 10:50:00.840650 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.840606    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3750a5b5-5b59-4bcf-b9f5-d07434945f20-metrics-certs\") pod \"maas-api-e2e-ait-d303059f-6f5fc74575-kll96\" (UID: \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"
Sep 19 10:50:00.846952 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.846930    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mgqm\" (UniqueName: \"kubernetes.io/projected/3750a5b5-5b59-4bcf-b9f5-d07434945f20-kube-api-access-7mgqm\") pod \"maas-api-e2e-ait-d303059f-6f5fc74575-kll96\" (UID: \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"
Sep 19 10:50:00.944295 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:00.944238    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"
Sep 19 10:50:01.222708 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:01.222678    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"]
Sep 19 10:50:01.224713 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:50:01.224680    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3750a5b5_5b59_4bcf_b9f5_d07434945f20.slice/crio-e238bd081d122ba64b7af7c8d437d6163be598b21a269b96159bbe60f320558c WatchSource:0}: Error finding container e238bd081d122ba64b7af7c8d437d6163be598b21a269b96159bbe60f320558c: Status 404 returned error can't find the container with id e238bd081d122ba64b7af7c8d437d6163be598b21a269b96159bbe60f320558c
Sep 19 10:50:01.355686 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:01.355646    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96" event={"ID":"3750a5b5-5b59-4bcf-b9f5-d07434945f20","Type":"ContainerStarted","Data":"e238bd081d122ba64b7af7c8d437d6163be598b21a269b96159bbe60f320558c"}
Sep 19 10:50:02.362911 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:02.362870    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96" event={"ID":"3750a5b5-5b59-4bcf-b9f5-d07434945f20","Type":"ContainerStarted","Data":"a18cb044d7917a010766d094f8b1c51a39dd10a6ab41a863a1faf78e994372e9"}
Sep 19 10:50:02.363322 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:02.363081    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"
Sep 19 10:50:02.377605 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:02.377554    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96" podStartSLOduration=1.901429522 podStartE2EDuration="2.377518374s" totalImagesPullingTime="476.088852ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:50:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:50:02.377291546 +0000 UTC m=+811.873597639" watchObservedRunningTime="2026-09-19 10:50:02.377518374 +0000 UTC m=+811.873824469"
Sep 19 10:50:08.374651 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:08.374621    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"
Sep 19 10:50:30.917137 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:30.917102    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"]
Sep 19 10:50:30.917735 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:30.917339    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96" podUID="3750a5b5-5b59-4bcf-b9f5-d07434945f20" containerName="maas-api" containerID="cri-o://a18cb044d7917a010766d094f8b1c51a39dd10a6ab41a863a1faf78e994372e9" gracePeriod=30
Sep 19 10:50:31.170475 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.170413    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"
Sep 19 10:50:31.224786 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.224758    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3750a5b5-5b59-4bcf-b9f5-d07434945f20-openshift-service-ca\") pod \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\" (UID: \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\") "
Sep 19 10:50:31.224986 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.224818    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3750a5b5-5b59-4bcf-b9f5-d07434945f20-metrics-certs\") pod \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\" (UID: \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\") "
Sep 19 10:50:31.224986 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.224928    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3750a5b5-5b59-4bcf-b9f5-d07434945f20-maas-api-tls\") pod \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\" (UID: \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\") "
Sep 19 10:50:31.224986 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.224955    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mgqm\" (UniqueName: \"kubernetes.io/projected/3750a5b5-5b59-4bcf-b9f5-d07434945f20-kube-api-access-7mgqm\") pod \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\" (UID: \"3750a5b5-5b59-4bcf-b9f5-d07434945f20\") "
Sep 19 10:50:31.225222 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.225099    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3750a5b5-5b59-4bcf-b9f5-d07434945f20-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3750a5b5-5b59-4bcf-b9f5-d07434945f20" (UID: "3750a5b5-5b59-4bcf-b9f5-d07434945f20"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:50:31.225336 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.225317    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3750a5b5-5b59-4bcf-b9f5-d07434945f20-openshift-service-ca\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:50:31.227340 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.227315    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3750a5b5-5b59-4bcf-b9f5-d07434945f20-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3750a5b5-5b59-4bcf-b9f5-d07434945f20" (UID: "3750a5b5-5b59-4bcf-b9f5-d07434945f20"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:50:31.227436 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.227384    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3750a5b5-5b59-4bcf-b9f5-d07434945f20-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3750a5b5-5b59-4bcf-b9f5-d07434945f20" (UID: "3750a5b5-5b59-4bcf-b9f5-d07434945f20"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:50:31.227526 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.227509    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3750a5b5-5b59-4bcf-b9f5-d07434945f20-kube-api-access-7mgqm" (OuterVolumeSpecName: "kube-api-access-7mgqm") pod "3750a5b5-5b59-4bcf-b9f5-d07434945f20" (UID: "3750a5b5-5b59-4bcf-b9f5-d07434945f20"). InnerVolumeSpecName "kube-api-access-7mgqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:50:31.326121 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.326085    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3750a5b5-5b59-4bcf-b9f5-d07434945f20-maas-api-tls\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:50:31.326121 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.326116    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mgqm\" (UniqueName: \"kubernetes.io/projected/3750a5b5-5b59-4bcf-b9f5-d07434945f20-kube-api-access-7mgqm\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:50:31.326121 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.326127    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3750a5b5-5b59-4bcf-b9f5-d07434945f20-metrics-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:50:31.524154 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.524067    2576 generic.go:356] "Generic (PLEG): container finished" podID="3750a5b5-5b59-4bcf-b9f5-d07434945f20" containerID="a18cb044d7917a010766d094f8b1c51a39dd10a6ab41a863a1faf78e994372e9" exitCode=0
Sep 19 10:50:31.524154 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.524104    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"
Sep 19 10:50:31.524414 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.524105    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96" event={"ID":"3750a5b5-5b59-4bcf-b9f5-d07434945f20","Type":"ContainerDied","Data":"a18cb044d7917a010766d094f8b1c51a39dd10a6ab41a863a1faf78e994372e9"}
Sep 19 10:50:31.524414 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.524234    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96" event={"ID":"3750a5b5-5b59-4bcf-b9f5-d07434945f20","Type":"ContainerDied","Data":"e238bd081d122ba64b7af7c8d437d6163be598b21a269b96159bbe60f320558c"}
Sep 19 10:50:31.524414 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.524266    2576 scope.go:117] "RemoveContainer" containerID="a18cb044d7917a010766d094f8b1c51a39dd10a6ab41a863a1faf78e994372e9"
Sep 19 10:50:31.534912 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.534894    2576 scope.go:117] "RemoveContainer" containerID="a18cb044d7917a010766d094f8b1c51a39dd10a6ab41a863a1faf78e994372e9"
Sep 19 10:50:31.535206 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:50:31.535190    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a18cb044d7917a010766d094f8b1c51a39dd10a6ab41a863a1faf78e994372e9\": container with ID starting with a18cb044d7917a010766d094f8b1c51a39dd10a6ab41a863a1faf78e994372e9 not found: ID does not exist" containerID="a18cb044d7917a010766d094f8b1c51a39dd10a6ab41a863a1faf78e994372e9"
Sep 19 10:50:31.535262 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.535215    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a18cb044d7917a010766d094f8b1c51a39dd10a6ab41a863a1faf78e994372e9"} err="failed to get container status \"a18cb044d7917a010766d094f8b1c51a39dd10a6ab41a863a1faf78e994372e9\": rpc error: code = NotFound desc = could not find container \"a18cb044d7917a010766d094f8b1c51a39dd10a6ab41a863a1faf78e994372e9\": container with ID starting with a18cb044d7917a010766d094f8b1c51a39dd10a6ab41a863a1faf78e994372e9 not found: ID does not exist"
Sep 19 10:50:31.546639 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.546613    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"]
Sep 19 10:50:31.548712 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:31.548692    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d303059f-6f5fc74575-kll96"]
Sep 19 10:50:33.012870 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:33.012838    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3750a5b5-5b59-4bcf-b9f5-d07434945f20" path="/var/lib/kubelet/pods/3750a5b5-5b59-4bcf-b9f5-d07434945f20/volumes"
Sep 19 10:50:39.613342 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:39.613306    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"]
Sep 19 10:50:39.614036 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:39.614016    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3750a5b5-5b59-4bcf-b9f5-d07434945f20" containerName="maas-api"
Sep 19 10:50:39.614161 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:39.614057    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3750a5b5-5b59-4bcf-b9f5-d07434945f20" containerName="maas-api"
Sep 19 10:50:39.614228 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:39.614187    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3750a5b5-5b59-4bcf-b9f5-d07434945f20" containerName="maas-api"
Sep 19 10:50:39.618596 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:39.618576    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"
Sep 19 10:50:39.620926 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:39.620902    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-3ee7a92b-serving-cert\""
Sep 19 10:50:39.625615 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:39.625589    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"]
Sep 19 10:50:39.707743 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:39.707713    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28cc76df-456b-4792-b144-61390b418e81-maas-api-tls\") pod \"maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb\" (UID: \"28cc76df-456b-4792-b144-61390b418e81\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"
Sep 19 10:50:39.707873 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:39.707757    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w88lx\" (UniqueName: \"kubernetes.io/projected/28cc76df-456b-4792-b144-61390b418e81-kube-api-access-w88lx\") pod \"maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb\" (UID: \"28cc76df-456b-4792-b144-61390b418e81\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"
Sep 19 10:50:39.707873 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:39.707786    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28cc76df-456b-4792-b144-61390b418e81-openshift-service-ca\") pod \"maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb\" (UID: \"28cc76df-456b-4792-b144-61390b418e81\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"
Sep 19 10:50:39.707873 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:39.707821    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28cc76df-456b-4792-b144-61390b418e81-metrics-certs\") pod \"maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb\" (UID: \"28cc76df-456b-4792-b144-61390b418e81\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"
Sep 19 10:50:39.809179 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:39.809142    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28cc76df-456b-4792-b144-61390b418e81-maas-api-tls\") pod \"maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb\" (UID: \"28cc76df-456b-4792-b144-61390b418e81\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"
Sep 19 10:50:39.809384 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:39.809197    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w88lx\" (UniqueName: \"kubernetes.io/projected/28cc76df-456b-4792-b144-61390b418e81-kube-api-access-w88lx\") pod \"maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb\" (UID: \"28cc76df-456b-4792-b144-61390b418e81\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"
Sep 19 10:50:39.809384 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:39.809239    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28cc76df-456b-4792-b144-61390b418e81-openshift-service-ca\") pod \"maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb\" (UID: \"28cc76df-456b-4792-b144-61390b418e81\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"
Sep 19 10:50:39.809384 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:39.809256    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28cc76df-456b-4792-b144-61390b418e81-metrics-certs\") pod \"maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb\" (UID: \"28cc76df-456b-4792-b144-61390b418e81\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"
Sep 19 10:50:39.809384 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:50:39.809304    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-serving-cert: secret "maas-api-e2e-derive-3ee7a92b-serving-cert" not found
Sep 19 10:50:39.809609 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:50:39.809390    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28cc76df-456b-4792-b144-61390b418e81-maas-api-tls podName:28cc76df-456b-4792-b144-61390b418e81 nodeName:}" failed. No retries permitted until 2026-09-19 10:50:40.309371831 +0000 UTC m=+849.805677903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/28cc76df-456b-4792-b144-61390b418e81-maas-api-tls") pod "maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb" (UID: "28cc76df-456b-4792-b144-61390b418e81") : secret "maas-api-e2e-derive-3ee7a92b-serving-cert" not found
Sep 19 10:50:39.810032 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:39.810003    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28cc76df-456b-4792-b144-61390b418e81-openshift-service-ca\") pod \"maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb\" (UID: \"28cc76df-456b-4792-b144-61390b418e81\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"
Sep 19 10:50:39.812084 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:39.812064    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28cc76df-456b-4792-b144-61390b418e81-metrics-certs\") pod \"maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb\" (UID: \"28cc76df-456b-4792-b144-61390b418e81\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"
Sep 19 10:50:39.817799 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:39.817777    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w88lx\" (UniqueName: \"kubernetes.io/projected/28cc76df-456b-4792-b144-61390b418e81-kube-api-access-w88lx\") pod \"maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb\" (UID: \"28cc76df-456b-4792-b144-61390b418e81\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"
Sep 19 10:50:40.314283 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:40.314250    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28cc76df-456b-4792-b144-61390b418e81-maas-api-tls\") pod \"maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb\" (UID: \"28cc76df-456b-4792-b144-61390b418e81\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"
Sep 19 10:50:40.316702 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:40.316669    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28cc76df-456b-4792-b144-61390b418e81-maas-api-tls\") pod \"maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb\" (UID: \"28cc76df-456b-4792-b144-61390b418e81\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"
Sep 19 10:50:40.533928 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:40.533895    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"
Sep 19 10:50:40.808007 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:40.807984    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"]
Sep 19 10:50:40.810473 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:50:40.810447    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28cc76df_456b_4792_b144_61390b418e81.slice/crio-c79efa0701f0255cd5def4525065fbf389d7634b2df0f2cbbeea16ceed667280 WatchSource:0}: Error finding container c79efa0701f0255cd5def4525065fbf389d7634b2df0f2cbbeea16ceed667280: Status 404 returned error can't find the container with id c79efa0701f0255cd5def4525065fbf389d7634b2df0f2cbbeea16ceed667280
Sep 19 10:50:41.583242 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:41.583167    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb" event={"ID":"28cc76df-456b-4792-b144-61390b418e81","Type":"ContainerStarted","Data":"d7060d2926e30122add043845cb3e07700dca3c3bbdb9902178faf8efec90975"}
Sep 19 10:50:41.583242 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:41.583213    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb" event={"ID":"28cc76df-456b-4792-b144-61390b418e81","Type":"ContainerStarted","Data":"c79efa0701f0255cd5def4525065fbf389d7634b2df0f2cbbeea16ceed667280"}
Sep 19 10:50:41.583414 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:41.583258    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"
Sep 19 10:50:41.598766 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:41.598720    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb" podStartSLOduration=2.263099703 podStartE2EDuration="2.598707147s" totalImagesPullingTime="335.607444ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:50:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:50:41.598096719 +0000 UTC m=+851.094402813" watchObservedRunningTime="2026-09-19 10:50:41.598707147 +0000 UTC m=+851.095013240"
Sep 19 10:50:47.595142 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:50:47.595115    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"
Sep 19 10:51:16.963632 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:16.963600    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"]
Sep 19 10:51:16.964117 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:16.963819    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb" podUID="28cc76df-456b-4792-b144-61390b418e81" containerName="maas-api" containerID="cri-o://d7060d2926e30122add043845cb3e07700dca3c3bbdb9902178faf8efec90975" gracePeriod=30
Sep 19 10:51:17.221769 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.221708    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"
Sep 19 10:51:17.355839 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.355805    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28cc76df-456b-4792-b144-61390b418e81-maas-api-tls\") pod \"28cc76df-456b-4792-b144-61390b418e81\" (UID: \"28cc76df-456b-4792-b144-61390b418e81\") "
Sep 19 10:51:17.356035 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.355869    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28cc76df-456b-4792-b144-61390b418e81-metrics-certs\") pod \"28cc76df-456b-4792-b144-61390b418e81\" (UID: \"28cc76df-456b-4792-b144-61390b418e81\") "
Sep 19 10:51:17.356035 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.355985    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w88lx\" (UniqueName: \"kubernetes.io/projected/28cc76df-456b-4792-b144-61390b418e81-kube-api-access-w88lx\") pod \"28cc76df-456b-4792-b144-61390b418e81\" (UID: \"28cc76df-456b-4792-b144-61390b418e81\") "
Sep 19 10:51:17.356218 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.356084    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28cc76df-456b-4792-b144-61390b418e81-openshift-service-ca\") pod \"28cc76df-456b-4792-b144-61390b418e81\" (UID: \"28cc76df-456b-4792-b144-61390b418e81\") "
Sep 19 10:51:17.356729 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.356685    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28cc76df-456b-4792-b144-61390b418e81-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "28cc76df-456b-4792-b144-61390b418e81" (UID: "28cc76df-456b-4792-b144-61390b418e81"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:51:17.358391 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.358360    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28cc76df-456b-4792-b144-61390b418e81-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "28cc76df-456b-4792-b144-61390b418e81" (UID: "28cc76df-456b-4792-b144-61390b418e81"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:51:17.358950 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.358928    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28cc76df-456b-4792-b144-61390b418e81-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "28cc76df-456b-4792-b144-61390b418e81" (UID: "28cc76df-456b-4792-b144-61390b418e81"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:51:17.359022 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.358954    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28cc76df-456b-4792-b144-61390b418e81-kube-api-access-w88lx" (OuterVolumeSpecName: "kube-api-access-w88lx") pod "28cc76df-456b-4792-b144-61390b418e81" (UID: "28cc76df-456b-4792-b144-61390b418e81"). InnerVolumeSpecName "kube-api-access-w88lx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:51:17.457463 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.457429    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w88lx\" (UniqueName: \"kubernetes.io/projected/28cc76df-456b-4792-b144-61390b418e81-kube-api-access-w88lx\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:51:17.457463 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.457460    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28cc76df-456b-4792-b144-61390b418e81-openshift-service-ca\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:51:17.457666 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.457473    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28cc76df-456b-4792-b144-61390b418e81-maas-api-tls\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:51:17.457666 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.457485    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28cc76df-456b-4792-b144-61390b418e81-metrics-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:51:17.799235 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.799203    2576 generic.go:356] "Generic (PLEG): container finished" podID="28cc76df-456b-4792-b144-61390b418e81" containerID="d7060d2926e30122add043845cb3e07700dca3c3bbdb9902178faf8efec90975" exitCode=0
Sep 19 10:51:17.799406 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.799246    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"
Sep 19 10:51:17.799406 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.799292    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb" event={"ID":"28cc76df-456b-4792-b144-61390b418e81","Type":"ContainerDied","Data":"d7060d2926e30122add043845cb3e07700dca3c3bbdb9902178faf8efec90975"}
Sep 19 10:51:17.799406 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.799345    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb" event={"ID":"28cc76df-456b-4792-b144-61390b418e81","Type":"ContainerDied","Data":"c79efa0701f0255cd5def4525065fbf389d7634b2df0f2cbbeea16ceed667280"}
Sep 19 10:51:17.799406 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.799366    2576 scope.go:117] "RemoveContainer" containerID="d7060d2926e30122add043845cb3e07700dca3c3bbdb9902178faf8efec90975"
Sep 19 10:51:17.809307 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.809289    2576 scope.go:117] "RemoveContainer" containerID="d7060d2926e30122add043845cb3e07700dca3c3bbdb9902178faf8efec90975"
Sep 19 10:51:17.809584 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:51:17.809565    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7060d2926e30122add043845cb3e07700dca3c3bbdb9902178faf8efec90975\": container with ID starting with d7060d2926e30122add043845cb3e07700dca3c3bbdb9902178faf8efec90975 not found: ID does not exist" containerID="d7060d2926e30122add043845cb3e07700dca3c3bbdb9902178faf8efec90975"
Sep 19 10:51:17.809658 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.809596    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7060d2926e30122add043845cb3e07700dca3c3bbdb9902178faf8efec90975"} err="failed to get container status \"d7060d2926e30122add043845cb3e07700dca3c3bbdb9902178faf8efec90975\": rpc error: code = NotFound desc = could not find container \"d7060d2926e30122add043845cb3e07700dca3c3bbdb9902178faf8efec90975\": container with ID starting with d7060d2926e30122add043845cb3e07700dca3c3bbdb9902178faf8efec90975 not found: ID does not exist"
Sep 19 10:51:17.820971 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.820946    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"]
Sep 19 10:51:17.824863 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:17.824844    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3ee7a92b-8644c5ddb8-t7zzb"]
Sep 19 10:51:19.001822 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:19.001791    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28cc76df-456b-4792-b144-61390b418e81" path="/var/lib/kubelet/pods/28cc76df-456b-4792-b144-61390b418e81/volumes"
Sep 19 10:51:30.959602 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:30.959521    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 10:51:30.961559 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:30.961540    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 10:51:53.860707 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:53.860676    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"]
Sep 19 10:51:53.861336 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:53.860878    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz" podUID="0b6dc030-a71c-465c-8788-349970253cb6" containerName="maas-api" containerID="cri-o://7c5cd883a946f2ad9e363bbb8266dbfef73affde7229c255eab5462d5a11cc81" gracePeriod=30
Sep 19 10:51:53.918974 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:53.918936    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz" podUID="0b6dc030-a71c-465c-8788-349970253cb6" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.51:8443/health\": dial tcp 10.133.0.51:8443: connect: connection refused"
Sep 19 10:51:53.926064 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:53.926018    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c"]
Sep 19 10:51:53.926275 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:53.926234    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c" podUID="02d306ea-2c5b-41ae-bca9-dfacc4ea49e5" containerName="payload-pre-processing" containerID="cri-o://5a98f1f2842408481b83eb9562a0bc0e8179b5a4ee20e16890b56979cf7ff812" gracePeriod=30
Sep 19 10:51:54.002824 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.002796    2576 generic.go:356] "Generic (PLEG): container finished" podID="0b6dc030-a71c-465c-8788-349970253cb6" containerID="7c5cd883a946f2ad9e363bbb8266dbfef73affde7229c255eab5462d5a11cc81" exitCode=0
Sep 19 10:51:54.002977 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.002869    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz" event={"ID":"0b6dc030-a71c-465c-8788-349970253cb6","Type":"ContainerDied","Data":"7c5cd883a946f2ad9e363bbb8266dbfef73affde7229c255eab5462d5a11cc81"}
Sep 19 10:51:54.218030 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.218005    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"
Sep 19 10:51:54.221260 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.221237    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c"
Sep 19 10:51:54.292761 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.292732    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b6dc030-a71c-465c-8788-349970253cb6-openshift-service-ca\") pod \"0b6dc030-a71c-465c-8788-349970253cb6\" (UID: \"0b6dc030-a71c-465c-8788-349970253cb6\") "
Sep 19 10:51:54.292892 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.292780    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b6dc030-a71c-465c-8788-349970253cb6-maas-api-tls\") pod \"0b6dc030-a71c-465c-8788-349970253cb6\" (UID: \"0b6dc030-a71c-465c-8788-349970253cb6\") "
Sep 19 10:51:54.292892 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.292817    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hl2ts\" (UniqueName: \"kubernetes.io/projected/0b6dc030-a71c-465c-8788-349970253cb6-kube-api-access-hl2ts\") pod \"0b6dc030-a71c-465c-8788-349970253cb6\" (UID: \"0b6dc030-a71c-465c-8788-349970253cb6\") "
Sep 19 10:51:54.292892 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.292836    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02d306ea-2c5b-41ae-bca9-dfacc4ea49e5-plugins-config-volume\") pod \"02d306ea-2c5b-41ae-bca9-dfacc4ea49e5\" (UID: \"02d306ea-2c5b-41ae-bca9-dfacc4ea49e5\") "
Sep 19 10:51:54.292892 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.292853    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b6dc030-a71c-465c-8788-349970253cb6-metrics-certs\") pod \"0b6dc030-a71c-465c-8788-349970253cb6\" (UID: \"0b6dc030-a71c-465c-8788-349970253cb6\") "
Sep 19 10:51:54.292892 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.292869    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hm8r9\" (UniqueName: \"kubernetes.io/projected/02d306ea-2c5b-41ae-bca9-dfacc4ea49e5-kube-api-access-hm8r9\") pod \"02d306ea-2c5b-41ae-bca9-dfacc4ea49e5\" (UID: \"02d306ea-2c5b-41ae-bca9-dfacc4ea49e5\") "
Sep 19 10:51:54.293192 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.293158    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b6dc030-a71c-465c-8788-349970253cb6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0b6dc030-a71c-465c-8788-349970253cb6" (UID: "0b6dc030-a71c-465c-8788-349970253cb6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:51:54.293256 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.293235    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02d306ea-2c5b-41ae-bca9-dfacc4ea49e5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "02d306ea-2c5b-41ae-bca9-dfacc4ea49e5" (UID: "02d306ea-2c5b-41ae-bca9-dfacc4ea49e5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:51:54.293318 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.293261    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b6dc030-a71c-465c-8788-349970253cb6-openshift-service-ca\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:51:54.295273 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.295250    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b6dc030-a71c-465c-8788-349970253cb6-kube-api-access-hl2ts" (OuterVolumeSpecName: "kube-api-access-hl2ts") pod "0b6dc030-a71c-465c-8788-349970253cb6" (UID: "0b6dc030-a71c-465c-8788-349970253cb6"). InnerVolumeSpecName "kube-api-access-hl2ts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:51:54.295390 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.295284    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02d306ea-2c5b-41ae-bca9-dfacc4ea49e5-kube-api-access-hm8r9" (OuterVolumeSpecName: "kube-api-access-hm8r9") pod "02d306ea-2c5b-41ae-bca9-dfacc4ea49e5" (UID: "02d306ea-2c5b-41ae-bca9-dfacc4ea49e5"). InnerVolumeSpecName "kube-api-access-hm8r9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:51:54.295450 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.295436    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b6dc030-a71c-465c-8788-349970253cb6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0b6dc030-a71c-465c-8788-349970253cb6" (UID: "0b6dc030-a71c-465c-8788-349970253cb6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:51:54.295546 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.295532    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b6dc030-a71c-465c-8788-349970253cb6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0b6dc030-a71c-465c-8788-349970253cb6" (UID: "0b6dc030-a71c-465c-8788-349970253cb6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:51:54.394744 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.394681    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b6dc030-a71c-465c-8788-349970253cb6-maas-api-tls\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:51:54.394744 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.394703    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl2ts\" (UniqueName: \"kubernetes.io/projected/0b6dc030-a71c-465c-8788-349970253cb6-kube-api-access-hl2ts\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:51:54.394744 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.394715    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02d306ea-2c5b-41ae-bca9-dfacc4ea49e5-plugins-config-volume\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:51:54.394744 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.394723    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0b6dc030-a71c-465c-8788-349970253cb6-metrics-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:51:54.394744 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:54.394731    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hm8r9\" (UniqueName: \"kubernetes.io/projected/02d306ea-2c5b-41ae-bca9-dfacc4ea49e5-kube-api-access-hm8r9\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:51:55.009311 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:55.009278    2576 generic.go:356] "Generic (PLEG): container finished" podID="02d306ea-2c5b-41ae-bca9-dfacc4ea49e5" containerID="5a98f1f2842408481b83eb9562a0bc0e8179b5a4ee20e16890b56979cf7ff812" exitCode=0
Sep 19 10:51:55.009705 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:55.009321    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c"
Sep 19 10:51:55.009705 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:55.009363    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c" event={"ID":"02d306ea-2c5b-41ae-bca9-dfacc4ea49e5","Type":"ContainerDied","Data":"5a98f1f2842408481b83eb9562a0bc0e8179b5a4ee20e16890b56979cf7ff812"}
Sep 19 10:51:55.009705 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:55.009426    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c" event={"ID":"02d306ea-2c5b-41ae-bca9-dfacc4ea49e5","Type":"ContainerDied","Data":"7c5da31b4c98f4992f93c56ee94ebebaacb01971954f212ddd5471be8fcd7e3f"}
Sep 19 10:51:55.009705 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:55.009458    2576 scope.go:117] "RemoveContainer" containerID="5a98f1f2842408481b83eb9562a0bc0e8179b5a4ee20e16890b56979cf7ff812"
Sep 19 10:51:55.011189 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:55.010996    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"
Sep 19 10:51:55.011189 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:55.011084    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz" event={"ID":"0b6dc030-a71c-465c-8788-349970253cb6","Type":"ContainerDied","Data":"2d372b692767eace235abb0b52eb8b2bb0bc886cdff6a4fcaf5243bda0f00b39"}
Sep 19 10:51:55.020379 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:55.020359    2576 scope.go:117] "RemoveContainer" containerID="5a98f1f2842408481b83eb9562a0bc0e8179b5a4ee20e16890b56979cf7ff812"
Sep 19 10:51:55.020657 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:51:55.020638    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a98f1f2842408481b83eb9562a0bc0e8179b5a4ee20e16890b56979cf7ff812\": container with ID starting with 5a98f1f2842408481b83eb9562a0bc0e8179b5a4ee20e16890b56979cf7ff812 not found: ID does not exist" containerID="5a98f1f2842408481b83eb9562a0bc0e8179b5a4ee20e16890b56979cf7ff812"
Sep 19 10:51:55.020740 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:55.020664    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a98f1f2842408481b83eb9562a0bc0e8179b5a4ee20e16890b56979cf7ff812"} err="failed to get container status \"5a98f1f2842408481b83eb9562a0bc0e8179b5a4ee20e16890b56979cf7ff812\": rpc error: code = NotFound desc = could not find container \"5a98f1f2842408481b83eb9562a0bc0e8179b5a4ee20e16890b56979cf7ff812\": container with ID starting with 5a98f1f2842408481b83eb9562a0bc0e8179b5a4ee20e16890b56979cf7ff812 not found: ID does not exist"
Sep 19 10:51:55.020740 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:55.020682    2576 scope.go:117] "RemoveContainer" containerID="7c5cd883a946f2ad9e363bbb8266dbfef73affde7229c255eab5462d5a11cc81"
Sep 19 10:51:55.030912 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:55.030889    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"]
Sep 19 10:51:55.032949 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:55.032930    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-7bcaa9-64fcd875c5-wkrwz"]
Sep 19 10:51:55.042542 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:55.042521    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c"]
Sep 19 10:51:55.048598 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:55.048576    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-7bcaa9-85b89979f8-z9r5c"]
Sep 19 10:51:57.001798 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:57.001767    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02d306ea-2c5b-41ae-bca9-dfacc4ea49e5" path="/var/lib/kubelet/pods/02d306ea-2c5b-41ae-bca9-dfacc4ea49e5/volumes"
Sep 19 10:51:57.002180 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:51:57.002158    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b6dc030-a71c-465c-8788-349970253cb6" path="/var/lib/kubelet/pods/0b6dc030-a71c-465c-8788-349970253cb6/volumes"
Sep 19 10:52:06.082363 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.082320    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"]
Sep 19 10:52:06.082825 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.082807    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b6dc030-a71c-465c-8788-349970253cb6" containerName="maas-api"
Sep 19 10:52:06.082901 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.082828    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b6dc030-a71c-465c-8788-349970253cb6" containerName="maas-api"
Sep 19 10:52:06.082901 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.082848    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28cc76df-456b-4792-b144-61390b418e81" containerName="maas-api"
Sep 19 10:52:06.082901 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.082853    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="28cc76df-456b-4792-b144-61390b418e81" containerName="maas-api"
Sep 19 10:52:06.082901 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.082860    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02d306ea-2c5b-41ae-bca9-dfacc4ea49e5" containerName="payload-pre-processing"
Sep 19 10:52:06.082901 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.082865    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="02d306ea-2c5b-41ae-bca9-dfacc4ea49e5" containerName="payload-pre-processing"
Sep 19 10:52:06.083121 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.082938    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="02d306ea-2c5b-41ae-bca9-dfacc4ea49e5" containerName="payload-pre-processing"
Sep 19 10:52:06.083121 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.082944    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b6dc030-a71c-465c-8788-349970253cb6" containerName="maas-api"
Sep 19 10:52:06.083121 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.082953    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="28cc76df-456b-4792-b144-61390b418e81" containerName="maas-api"
Sep 19 10:52:06.087306 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.087290    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.089839 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.089818    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-e898f8-openshift-default-dockercfg-ksbk7\""
Sep 19 10:52:06.097615 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.097579    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"]
Sep 19 10:52:06.199190 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.199157    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cadfa4e9-7d31-487d-8b57-4087c39c917a-istiod-ca-cert\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.199339 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.199204    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-envoy\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.199339 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.199274    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-workload-certs\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.199339 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.199333    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-data\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.199451 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.199358    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjdbn\" (UniqueName: \"kubernetes.io/projected/cadfa4e9-7d31-487d-8b57-4087c39c917a-kube-api-access-bjdbn\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.199451 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.199375    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-token\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.199451 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.199393    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-workload-socket\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.199551 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.199470    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-podinfo\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.199551 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.199493    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-credential-socket\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.300408 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.300375    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-data\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.300408 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.300410    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjdbn\" (UniqueName: \"kubernetes.io/projected/cadfa4e9-7d31-487d-8b57-4087c39c917a-kube-api-access-bjdbn\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.300594 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.300427    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-token\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.300594 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.300443    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-workload-socket\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.300594 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.300495    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-podinfo\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.300594 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.300514    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-credential-socket\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.300594 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.300577    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cadfa4e9-7d31-487d-8b57-4087c39c917a-istiod-ca-cert\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.300840 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.300726    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-envoy\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.300840 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.300770    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-workload-certs\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.300840 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.300799    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-data\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.300998 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.300953    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-workload-socket\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.301088 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.301004    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-credential-socket\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.301207 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.301181    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-workload-certs\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.301334 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.301298    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cadfa4e9-7d31-487d-8b57-4087c39c917a-istiod-ca-cert\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.303182 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.303158    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-envoy\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.303266 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.303234    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-podinfo\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.311449 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.311427    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-token\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.311864 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.311843    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjdbn\" (UniqueName: \"kubernetes.io/projected/cadfa4e9-7d31-487d-8b57-4087c39c917a-kube-api-access-bjdbn\") pod \"e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") " pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.399849 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.399828    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:06.676896 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.676867    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"]
Sep 19 10:52:06.678153 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:52:06.678125    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcadfa4e9_7d31_487d_8b57_4087c39c917a.slice/crio-e6c08216e36ab2b04ba1fa9d59f77a20f623054ddd61b74c6cde6ed52cee0b9c WatchSource:0}: Error finding container e6c08216e36ab2b04ba1fa9d59f77a20f623054ddd61b74c6cde6ed52cee0b9c: Status 404 returned error can't find the container with id e6c08216e36ab2b04ba1fa9d59f77a20f623054ddd61b74c6cde6ed52cee0b9c
Sep 19 10:52:06.680383 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.680351    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:52:06.680473 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.680414    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:52:06.680473 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:06.680442    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:52:07.077120 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:07.076994    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg" event={"ID":"cadfa4e9-7d31-487d-8b57-4087c39c917a","Type":"ContainerStarted","Data":"5e286408bca4e0bc26339e4c017f8e622d152369a5c67a329a48699d5cc29f29"}
Sep 19 10:52:07.077120 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:07.077090    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg" event={"ID":"cadfa4e9-7d31-487d-8b57-4087c39c917a","Type":"ContainerStarted","Data":"e6c08216e36ab2b04ba1fa9d59f77a20f623054ddd61b74c6cde6ed52cee0b9c"}
Sep 19 10:52:07.095072 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:07.095009    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg" podStartSLOduration=1.094994943 podStartE2EDuration="1.094994943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:52:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:52:07.09379995 +0000 UTC m=+936.590106042" watchObservedRunningTime="2026-09-19 10:52:07.094994943 +0000 UTC m=+936.591301034"
Sep 19 10:52:07.400556 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:07.400526    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:07.406103 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:07.406076    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:08.082370 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:08.082341    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:08.083745 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:08.083728    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:11.680352 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:11.680314    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"]
Sep 19 10:52:11.684414 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:11.684390    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"
Sep 19 10:52:11.686569 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:11.686552    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-e898f8-serving-cert\""
Sep 19 10:52:11.695350 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:11.695328    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"]
Sep 19 10:52:11.853404 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:11.853373    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70af8e0f-55de-411e-80a9-dd921e23d99c-maas-api-tls\") pod \"maas-api-e2e-explicit-e898f8-59d5b95679-djf8v\" (UID: \"70af8e0f-55de-411e-80a9-dd921e23d99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"
Sep 19 10:52:11.853541 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:11.853414    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70af8e0f-55de-411e-80a9-dd921e23d99c-openshift-service-ca\") pod \"maas-api-e2e-explicit-e898f8-59d5b95679-djf8v\" (UID: \"70af8e0f-55de-411e-80a9-dd921e23d99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"
Sep 19 10:52:11.853541 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:11.853500    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70af8e0f-55de-411e-80a9-dd921e23d99c-metrics-certs\") pod \"maas-api-e2e-explicit-e898f8-59d5b95679-djf8v\" (UID: \"70af8e0f-55de-411e-80a9-dd921e23d99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"
Sep 19 10:52:11.853619 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:11.853549    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqbhq\" (UniqueName: \"kubernetes.io/projected/70af8e0f-55de-411e-80a9-dd921e23d99c-kube-api-access-jqbhq\") pod \"maas-api-e2e-explicit-e898f8-59d5b95679-djf8v\" (UID: \"70af8e0f-55de-411e-80a9-dd921e23d99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"
Sep 19 10:52:11.954663 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:11.954589    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqbhq\" (UniqueName: \"kubernetes.io/projected/70af8e0f-55de-411e-80a9-dd921e23d99c-kube-api-access-jqbhq\") pod \"maas-api-e2e-explicit-e898f8-59d5b95679-djf8v\" (UID: \"70af8e0f-55de-411e-80a9-dd921e23d99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"
Sep 19 10:52:11.954804 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:11.954671    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70af8e0f-55de-411e-80a9-dd921e23d99c-maas-api-tls\") pod \"maas-api-e2e-explicit-e898f8-59d5b95679-djf8v\" (UID: \"70af8e0f-55de-411e-80a9-dd921e23d99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"
Sep 19 10:52:11.954804 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:11.954704    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70af8e0f-55de-411e-80a9-dd921e23d99c-openshift-service-ca\") pod \"maas-api-e2e-explicit-e898f8-59d5b95679-djf8v\" (UID: \"70af8e0f-55de-411e-80a9-dd921e23d99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"
Sep 19 10:52:11.954804 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:11.954782    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70af8e0f-55de-411e-80a9-dd921e23d99c-metrics-certs\") pod \"maas-api-e2e-explicit-e898f8-59d5b95679-djf8v\" (UID: \"70af8e0f-55de-411e-80a9-dd921e23d99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"
Sep 19 10:52:11.955496 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:11.955471    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70af8e0f-55de-411e-80a9-dd921e23d99c-openshift-service-ca\") pod \"maas-api-e2e-explicit-e898f8-59d5b95679-djf8v\" (UID: \"70af8e0f-55de-411e-80a9-dd921e23d99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"
Sep 19 10:52:11.957543 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:11.957520    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70af8e0f-55de-411e-80a9-dd921e23d99c-maas-api-tls\") pod \"maas-api-e2e-explicit-e898f8-59d5b95679-djf8v\" (UID: \"70af8e0f-55de-411e-80a9-dd921e23d99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"
Sep 19 10:52:11.957604 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:11.957520    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70af8e0f-55de-411e-80a9-dd921e23d99c-metrics-certs\") pod \"maas-api-e2e-explicit-e898f8-59d5b95679-djf8v\" (UID: \"70af8e0f-55de-411e-80a9-dd921e23d99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"
Sep 19 10:52:11.962112 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:11.962090    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqbhq\" (UniqueName: \"kubernetes.io/projected/70af8e0f-55de-411e-80a9-dd921e23d99c-kube-api-access-jqbhq\") pod \"maas-api-e2e-explicit-e898f8-59d5b95679-djf8v\" (UID: \"70af8e0f-55de-411e-80a9-dd921e23d99c\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"
Sep 19 10:52:12.000386 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:12.000365    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"
Sep 19 10:52:12.274420 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:12.274390    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"]
Sep 19 10:52:12.276394 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:52:12.276356    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70af8e0f_55de_411e_80a9_dd921e23d99c.slice/crio-b3006c2dea3c27bdaf6201c24298ae3247cbc708ecfa3e45f41d1a5bda5befc3 WatchSource:0}: Error finding container b3006c2dea3c27bdaf6201c24298ae3247cbc708ecfa3e45f41d1a5bda5befc3: Status 404 returned error can't find the container with id b3006c2dea3c27bdaf6201c24298ae3247cbc708ecfa3e45f41d1a5bda5befc3
Sep 19 10:52:13.117905 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:13.117874    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v" event={"ID":"70af8e0f-55de-411e-80a9-dd921e23d99c","Type":"ContainerStarted","Data":"466c9a137574a691dc7b14528ef0eee58684cd1f9da66d170d769720f711dfe9"}
Sep 19 10:52:13.118282 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:13.117922    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v" event={"ID":"70af8e0f-55de-411e-80a9-dd921e23d99c","Type":"ContainerStarted","Data":"b3006c2dea3c27bdaf6201c24298ae3247cbc708ecfa3e45f41d1a5bda5befc3"}
Sep 19 10:52:13.118282 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:13.117959    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"
Sep 19 10:52:13.133104 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:13.133060    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v" podStartSLOduration=1.7339327789999999 podStartE2EDuration="2.13302791s" totalImagesPullingTime="399.095131ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:52:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:52:13.132399464 +0000 UTC m=+942.628705552" watchObservedRunningTime="2026-09-19 10:52:13.13302791 +0000 UTC m=+942.629334002"
Sep 19 10:52:19.128835 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:19.128807    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"
Sep 19 10:52:31.215714 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:31.215679    2576 generic.go:356] "Generic (PLEG): container finished" podID="cadfa4e9-7d31-487d-8b57-4087c39c917a" containerID="5e286408bca4e0bc26339e4c017f8e622d152369a5c67a329a48699d5cc29f29" exitCode=0
Sep 19 10:52:31.216145 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:31.215727    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg" event={"ID":"cadfa4e9-7d31-487d-8b57-4087c39c917a","Type":"ContainerDied","Data":"5e286408bca4e0bc26339e4c017f8e622d152369a5c67a329a48699d5cc29f29"}
Sep 19 10:52:31.216250 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:31.216233    2576 scope.go:117] "RemoveContainer" containerID="5e286408bca4e0bc26339e4c017f8e622d152369a5c67a329a48699d5cc29f29"
Sep 19 10:52:31.217455 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:31.217423    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:52:31.217586 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:31.217502    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:52:31.217586 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:31.217540    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:52:32.223457 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:32.223423    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg" event={"ID":"cadfa4e9-7d31-487d-8b57-4087c39c917a","Type":"ContainerStarted","Data":"22b423d5db5117bad10919b25546cc93b4e7ce2c0e7cf313ecd55afaaa055b38"}
Sep 19 10:52:32.400793 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:32.400762    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:32.400979 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:32.400958    2576 patch_prober.go:28] interesting pod/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 19 10:52:32.401035 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:32.401005    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg" podUID="cadfa4e9-7d31-487d-8b57-4087c39c917a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 19 10:52:33.400389 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:33.400354    2576 patch_prober.go:28] interesting pod/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 19 10:52:33.400832 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:33.400405    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg" podUID="cadfa4e9-7d31-487d-8b57-4087c39c917a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 19 10:52:34.400911 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:34.400876    2576 patch_prober.go:28] interesting pod/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 19 10:52:34.401399 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:34.400939    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg" podUID="cadfa4e9-7d31-487d-8b57-4087c39c917a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 19 10:52:35.400175 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:35.400138    2576 patch_prober.go:28] interesting pod/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 19 10:52:35.400344 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:35.400193    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg" podUID="cadfa4e9-7d31-487d-8b57-4087c39c917a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 19 10:52:36.400632 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:36.400597    2576 patch_prober.go:28] interesting pod/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Sep 19 10:52:36.401124 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:36.400659    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg" podUID="cadfa4e9-7d31-487d-8b57-4087c39c917a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Sep 19 10:52:37.449972 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:37.449935    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:37.450396 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:37.450354    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:37.450989 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:37.450971    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:52:52.462867 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.462794    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"]
Sep 19 10:52:52.467418 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.467402    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.469618 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.469601    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-18444f28-openshift-default-dockercfg-rrzvd\""
Sep 19 10:52:52.478911 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.478887    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"]
Sep 19 10:52:52.647028 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.646993    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-istio-envoy\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.647254 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.647117    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-workload-certs\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.647254 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.647182    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-workload-socket\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.647254 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.647209    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eda19392-f366-4802-abb5-2079c705ebf0-istio-token\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.647399 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.647253    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eda19392-f366-4802-abb5-2079c705ebf0-istiod-ca-cert\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.647399 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.647281    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdn5x\" (UniqueName: \"kubernetes.io/projected/eda19392-f366-4802-abb5-2079c705ebf0-kube-api-access-pdn5x\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.647399 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.647333    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eda19392-f366-4802-abb5-2079c705ebf0-istio-podinfo\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.647399 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.647371    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-istio-data\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.647399 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.647388    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-credential-socket\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.748238 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.748165    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-workload-certs\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.748238 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.748206    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-workload-socket\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.748238 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.748226    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eda19392-f366-4802-abb5-2079c705ebf0-istio-token\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.748483 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.748245    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eda19392-f366-4802-abb5-2079c705ebf0-istiod-ca-cert\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.748483 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.748278    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdn5x\" (UniqueName: \"kubernetes.io/projected/eda19392-f366-4802-abb5-2079c705ebf0-kube-api-access-pdn5x\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.748483 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.748325    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eda19392-f366-4802-abb5-2079c705ebf0-istio-podinfo\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.748483 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.748388    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-istio-data\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.748483 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.748414    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-credential-socket\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.748483 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.748450    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-istio-envoy\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.748779 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.748588    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-workload-certs\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.748779 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.748640    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-workload-socket\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.748847 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.748815    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-istio-data\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.748889 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.748848    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-credential-socket\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.749057 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.749027    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eda19392-f366-4802-abb5-2079c705ebf0-istiod-ca-cert\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.750918 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.750896    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-istio-envoy\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.751072 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.751022    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eda19392-f366-4802-abb5-2079c705ebf0-istio-podinfo\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.756461 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.756441    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdn5x\" (UniqueName: \"kubernetes.io/projected/eda19392-f366-4802-abb5-2079c705ebf0-kube-api-access-pdn5x\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.756541 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.756528    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eda19392-f366-4802-abb5-2079c705ebf0-istio-token\") pod \"e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") " pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:52.778560 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:52.778539    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:53.058398 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:53.058373    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"]
Sep 19 10:52:53.060206 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:52:53.060176    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeda19392_f366_4802_abb5_2079c705ebf0.slice/crio-bd98a9a842da9db769a67e34aa7f16cef93d6f8d408a9b3393718a72172ec33c WatchSource:0}: Error finding container bd98a9a842da9db769a67e34aa7f16cef93d6f8d408a9b3393718a72172ec33c: Status 404 returned error can't find the container with id bd98a9a842da9db769a67e34aa7f16cef93d6f8d408a9b3393718a72172ec33c
Sep 19 10:52:53.062367 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:53.062331    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:52:53.062451 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:53.062395    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:52:53.062451 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:53.062423    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:52:53.371370 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:53.371332    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg" event={"ID":"eda19392-f366-4802-abb5-2079c705ebf0","Type":"ContainerStarted","Data":"5a94ae44b5a37853af01068bd85c14826e9db55c0054686d432ab0e0929e5640"}
Sep 19 10:52:53.371538 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:53.371434    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg" event={"ID":"eda19392-f366-4802-abb5-2079c705ebf0","Type":"ContainerStarted","Data":"bd98a9a842da9db769a67e34aa7f16cef93d6f8d408a9b3393718a72172ec33c"}
Sep 19 10:52:53.391618 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:53.391572    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg" podStartSLOduration=1.391557125 podStartE2EDuration="1.391557125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:52:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:52:53.38992817 +0000 UTC m=+982.886234265" watchObservedRunningTime="2026-09-19 10:52:53.391557125 +0000 UTC m=+982.887863261"
Sep 19 10:52:53.779664 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:53.779583    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:54.783903 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:54.783878    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:55.381989 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:55.381965    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:52:55.382891 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:52:55.382870    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:53:01.508119 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:01.508086    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"]
Sep 19 10:53:01.512252 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:01.512228    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"
Sep 19 10:53:01.515160 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:01.515124    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-18444f28-serving-cert\""
Sep 19 10:53:01.524293 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:01.524266    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"]
Sep 19 10:53:01.528659 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:01.527989    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e29ad04-c28b-4736-9354-0552788f872f-metrics-certs\") pod \"maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt\" (UID: \"3e29ad04-c28b-4736-9354-0552788f872f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"
Sep 19 10:53:01.528659 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:01.528061    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xksvm\" (UniqueName: \"kubernetes.io/projected/3e29ad04-c28b-4736-9354-0552788f872f-kube-api-access-xksvm\") pod \"maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt\" (UID: \"3e29ad04-c28b-4736-9354-0552788f872f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"
Sep 19 10:53:01.528659 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:01.528144    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e29ad04-c28b-4736-9354-0552788f872f-openshift-service-ca\") pod \"maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt\" (UID: \"3e29ad04-c28b-4736-9354-0552788f872f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"
Sep 19 10:53:01.528659 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:01.528186    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e29ad04-c28b-4736-9354-0552788f872f-maas-api-tls\") pod \"maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt\" (UID: \"3e29ad04-c28b-4736-9354-0552788f872f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"
Sep 19 10:53:01.629476 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:01.629439    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e29ad04-c28b-4736-9354-0552788f872f-openshift-service-ca\") pod \"maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt\" (UID: \"3e29ad04-c28b-4736-9354-0552788f872f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"
Sep 19 10:53:01.629686 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:01.629508    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e29ad04-c28b-4736-9354-0552788f872f-maas-api-tls\") pod \"maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt\" (UID: \"3e29ad04-c28b-4736-9354-0552788f872f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"
Sep 19 10:53:01.629686 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:01.629660    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e29ad04-c28b-4736-9354-0552788f872f-metrics-certs\") pod \"maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt\" (UID: \"3e29ad04-c28b-4736-9354-0552788f872f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"
Sep 19 10:53:01.629827 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:01.629709    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xksvm\" (UniqueName: \"kubernetes.io/projected/3e29ad04-c28b-4736-9354-0552788f872f-kube-api-access-xksvm\") pod \"maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt\" (UID: \"3e29ad04-c28b-4736-9354-0552788f872f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"
Sep 19 10:53:01.630168 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:53:01.630143    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-serving-cert: secret "maas-api-e2e-mt-18444f28-serving-cert" not found
Sep 19 10:53:01.630391 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:53:01.630377    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e29ad04-c28b-4736-9354-0552788f872f-maas-api-tls podName:3e29ad04-c28b-4736-9354-0552788f872f nodeName:}" failed. No retries permitted until 2026-09-19 10:53:02.130354466 +0000 UTC m=+991.626660546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3e29ad04-c28b-4736-9354-0552788f872f-maas-api-tls") pod "maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt" (UID: "3e29ad04-c28b-4736-9354-0552788f872f") : secret "maas-api-e2e-mt-18444f28-serving-cert" not found
Sep 19 10:53:01.630501 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:01.630438    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e29ad04-c28b-4736-9354-0552788f872f-openshift-service-ca\") pod \"maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt\" (UID: \"3e29ad04-c28b-4736-9354-0552788f872f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"
Sep 19 10:53:01.632956 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:01.632935    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e29ad04-c28b-4736-9354-0552788f872f-metrics-certs\") pod \"maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt\" (UID: \"3e29ad04-c28b-4736-9354-0552788f872f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"
Sep 19 10:53:01.642532 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:01.642515    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xksvm\" (UniqueName: \"kubernetes.io/projected/3e29ad04-c28b-4736-9354-0552788f872f-kube-api-access-xksvm\") pod \"maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt\" (UID: \"3e29ad04-c28b-4736-9354-0552788f872f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"
Sep 19 10:53:02.140529 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:02.139915    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e29ad04-c28b-4736-9354-0552788f872f-maas-api-tls\") pod \"maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt\" (UID: \"3e29ad04-c28b-4736-9354-0552788f872f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"
Sep 19 10:53:02.150639 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:02.150590    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e29ad04-c28b-4736-9354-0552788f872f-maas-api-tls\") pod \"maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt\" (UID: \"3e29ad04-c28b-4736-9354-0552788f872f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"
Sep 19 10:53:02.427732 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:02.427284    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"
Sep 19 10:53:02.712560 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:02.712537    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"]
Sep 19 10:53:02.714480 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:53:02.714447    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e29ad04_c28b_4736_9354_0552788f872f.slice/crio-d2505296e27e54785be8e5e8890fb720e658f01e99072df28e0f4aba91625fc8 WatchSource:0}: Error finding container d2505296e27e54785be8e5e8890fb720e658f01e99072df28e0f4aba91625fc8: Status 404 returned error can't find the container with id d2505296e27e54785be8e5e8890fb720e658f01e99072df28e0f4aba91625fc8
Sep 19 10:53:03.431638 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:03.431603    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt" event={"ID":"3e29ad04-c28b-4736-9354-0552788f872f","Type":"ContainerStarted","Data":"49bc8b5fb0749667de46e897149e59ab7a05f682b6ba9fb0463aefeebc3f9051"}
Sep 19 10:53:03.431820 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:03.431656    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt" event={"ID":"3e29ad04-c28b-4736-9354-0552788f872f","Type":"ContainerStarted","Data":"d2505296e27e54785be8e5e8890fb720e658f01e99072df28e0f4aba91625fc8"}
Sep 19 10:53:03.431820 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:03.431807    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"
Sep 19 10:53:03.449438 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:03.449388    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt" podStartSLOduration=2.143783302 podStartE2EDuration="2.44937442s" totalImagesPullingTime="305.591118ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:53:03.447226865 +0000 UTC m=+992.943532957" watchObservedRunningTime="2026-09-19 10:53:03.44937442 +0000 UTC m=+992.945680512"
Sep 19 10:53:09.442507 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:09.442475    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"
Sep 19 10:53:22.532962 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:22.532928    2576 generic.go:356] "Generic (PLEG): container finished" podID="eda19392-f366-4802-abb5-2079c705ebf0" containerID="5a94ae44b5a37853af01068bd85c14826e9db55c0054686d432ab0e0929e5640" exitCode=0
Sep 19 10:53:22.533498 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:22.533005    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg" event={"ID":"eda19392-f366-4802-abb5-2079c705ebf0","Type":"ContainerDied","Data":"5a94ae44b5a37853af01068bd85c14826e9db55c0054686d432ab0e0929e5640"}
Sep 19 10:53:22.533617 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:22.533599    2576 scope.go:117] "RemoveContainer" containerID="5a94ae44b5a37853af01068bd85c14826e9db55c0054686d432ab0e0929e5640"
Sep 19 10:53:22.534910 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:22.534883    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:53:22.534976 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:22.534949    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:53:22.535013 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:22.534981    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:53:23.544673 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:23.544626    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg" event={"ID":"eda19392-f366-4802-abb5-2079c705ebf0","Type":"ContainerStarted","Data":"8ec58f04240c8dd44f23a11240cffbde39fe787c618cd6bcc8936bff5b58088f"}
Sep 19 10:53:23.779061 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:23.779005    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:53:23.780393 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:23.780366    2576 patch_prober.go:28] interesting pod/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 19 10:53:23.780521 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:23.780417    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg" podUID="eda19392-f366-4802-abb5-2079c705ebf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 19 10:53:24.779677 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:24.779634    2576 patch_prober.go:28] interesting pod/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 19 10:53:24.780076 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:24.779705    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg" podUID="eda19392-f366-4802-abb5-2079c705ebf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 19 10:53:25.382785 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:25.382748    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:53:25.779170 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:25.779089    2576 patch_prober.go:28] interesting pod/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 19 10:53:25.779170 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:25.779151    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg" podUID="eda19392-f366-4802-abb5-2079c705ebf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 19 10:53:26.779744 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:26.779707    2576 patch_prober.go:28] interesting pod/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 19 10:53:26.780224 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:26.779777    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg" podUID="eda19392-f366-4802-abb5-2079c705ebf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 19 10:53:27.779510 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:27.779477    2576 patch_prober.go:28] interesting pod/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Sep 19 10:53:27.779693 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:27.779525    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg" podUID="eda19392-f366-4802-abb5-2079c705ebf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Sep 19 10:53:28.783251 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:28.783224    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:53:28.784392 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:28.784370    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:53:45.578923 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:45.578893    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"]
Sep 19 10:53:45.579374 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:45.579134    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt" podUID="3e29ad04-c28b-4736-9354-0552788f872f" containerName="maas-api" containerID="cri-o://49bc8b5fb0749667de46e897149e59ab7a05f682b6ba9fb0463aefeebc3f9051" gracePeriod=30
Sep 19 10:53:45.834399 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:45.834337    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"
Sep 19 10:53:45.870209 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:45.870176    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xksvm\" (UniqueName: \"kubernetes.io/projected/3e29ad04-c28b-4736-9354-0552788f872f-kube-api-access-xksvm\") pod \"3e29ad04-c28b-4736-9354-0552788f872f\" (UID: \"3e29ad04-c28b-4736-9354-0552788f872f\") "
Sep 19 10:53:45.870388 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:45.870246    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e29ad04-c28b-4736-9354-0552788f872f-metrics-certs\") pod \"3e29ad04-c28b-4736-9354-0552788f872f\" (UID: \"3e29ad04-c28b-4736-9354-0552788f872f\") "
Sep 19 10:53:45.870388 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:45.870301    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e29ad04-c28b-4736-9354-0552788f872f-openshift-service-ca\") pod \"3e29ad04-c28b-4736-9354-0552788f872f\" (UID: \"3e29ad04-c28b-4736-9354-0552788f872f\") "
Sep 19 10:53:45.870388 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:45.870342    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e29ad04-c28b-4736-9354-0552788f872f-maas-api-tls\") pod \"3e29ad04-c28b-4736-9354-0552788f872f\" (UID: \"3e29ad04-c28b-4736-9354-0552788f872f\") "
Sep 19 10:53:45.871721 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:45.871689    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e29ad04-c28b-4736-9354-0552788f872f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3e29ad04-c28b-4736-9354-0552788f872f" (UID: "3e29ad04-c28b-4736-9354-0552788f872f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:53:45.888077 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:45.887995    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e29ad04-c28b-4736-9354-0552788f872f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3e29ad04-c28b-4736-9354-0552788f872f" (UID: "3e29ad04-c28b-4736-9354-0552788f872f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:53:45.891111 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:45.891064    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e29ad04-c28b-4736-9354-0552788f872f-kube-api-access-xksvm" (OuterVolumeSpecName: "kube-api-access-xksvm") pod "3e29ad04-c28b-4736-9354-0552788f872f" (UID: "3e29ad04-c28b-4736-9354-0552788f872f"). InnerVolumeSpecName "kube-api-access-xksvm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:53:45.891319 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:45.891287    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e29ad04-c28b-4736-9354-0552788f872f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3e29ad04-c28b-4736-9354-0552788f872f" (UID: "3e29ad04-c28b-4736-9354-0552788f872f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:53:45.971632 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:45.971601    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xksvm\" (UniqueName: \"kubernetes.io/projected/3e29ad04-c28b-4736-9354-0552788f872f-kube-api-access-xksvm\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:53:45.971632 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:45.971629    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e29ad04-c28b-4736-9354-0552788f872f-metrics-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:53:45.971632 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:45.971639    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e29ad04-c28b-4736-9354-0552788f872f-openshift-service-ca\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:53:45.971900 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:45.971647    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e29ad04-c28b-4736-9354-0552788f872f-maas-api-tls\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:53:46.691087 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:46.691055    2576 generic.go:356] "Generic (PLEG): container finished" podID="3e29ad04-c28b-4736-9354-0552788f872f" containerID="49bc8b5fb0749667de46e897149e59ab7a05f682b6ba9fb0463aefeebc3f9051" exitCode=0
Sep 19 10:53:46.691483 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:46.691110    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"
Sep 19 10:53:46.691483 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:46.691120    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt" event={"ID":"3e29ad04-c28b-4736-9354-0552788f872f","Type":"ContainerDied","Data":"49bc8b5fb0749667de46e897149e59ab7a05f682b6ba9fb0463aefeebc3f9051"}
Sep 19 10:53:46.691483 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:46.691183    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt" event={"ID":"3e29ad04-c28b-4736-9354-0552788f872f","Type":"ContainerDied","Data":"d2505296e27e54785be8e5e8890fb720e658f01e99072df28e0f4aba91625fc8"}
Sep 19 10:53:46.691483 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:46.691204    2576 scope.go:117] "RemoveContainer" containerID="49bc8b5fb0749667de46e897149e59ab7a05f682b6ba9fb0463aefeebc3f9051"
Sep 19 10:53:46.702008 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:46.701985    2576 scope.go:117] "RemoveContainer" containerID="49bc8b5fb0749667de46e897149e59ab7a05f682b6ba9fb0463aefeebc3f9051"
Sep 19 10:53:46.702314 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:53:46.702286    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49bc8b5fb0749667de46e897149e59ab7a05f682b6ba9fb0463aefeebc3f9051\": container with ID starting with 49bc8b5fb0749667de46e897149e59ab7a05f682b6ba9fb0463aefeebc3f9051 not found: ID does not exist" containerID="49bc8b5fb0749667de46e897149e59ab7a05f682b6ba9fb0463aefeebc3f9051"
Sep 19 10:53:46.702383 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:46.702325    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49bc8b5fb0749667de46e897149e59ab7a05f682b6ba9fb0463aefeebc3f9051"} err="failed to get container status \"49bc8b5fb0749667de46e897149e59ab7a05f682b6ba9fb0463aefeebc3f9051\": rpc error: code = NotFound desc = could not find container \"49bc8b5fb0749667de46e897149e59ab7a05f682b6ba9fb0463aefeebc3f9051\": container with ID starting with 49bc8b5fb0749667de46e897149e59ab7a05f682b6ba9fb0463aefeebc3f9051 not found: ID does not exist"
Sep 19 10:53:46.714393 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:46.714372    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"]
Sep 19 10:53:46.717432 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:46.717412    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-18444f28-8585bddb7f-kl8zt"]
Sep 19 10:53:47.002192 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:47.002126    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e29ad04-c28b-4736-9354-0552788f872f" path="/var/lib/kubelet/pods/3e29ad04-c28b-4736-9354-0552788f872f/volumes"
Sep 19 10:53:52.110958 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:52.110927    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"]
Sep 19 10:53:52.111639 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:52.111181    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg" podUID="eda19392-f366-4802-abb5-2079c705ebf0" containerName="istio-proxy" containerID="cri-o://8ec58f04240c8dd44f23a11240cffbde39fe787c618cd6bcc8936bff5b58088f" gracePeriod=30
Sep 19 10:53:57.364431 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.364409    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:53:57.474172 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.474111    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eda19392-f366-4802-abb5-2079c705ebf0-istiod-ca-cert\") pod \"eda19392-f366-4802-abb5-2079c705ebf0\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") "
Sep 19 10:53:57.474172 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.474155    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eda19392-f366-4802-abb5-2079c705ebf0-istio-token\") pod \"eda19392-f366-4802-abb5-2079c705ebf0\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") "
Sep 19 10:53:57.474324 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.474186    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-credential-socket\") pod \"eda19392-f366-4802-abb5-2079c705ebf0\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") "
Sep 19 10:53:57.474324 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.474287    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdn5x\" (UniqueName: \"kubernetes.io/projected/eda19392-f366-4802-abb5-2079c705ebf0-kube-api-access-pdn5x\") pod \"eda19392-f366-4802-abb5-2079c705ebf0\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") "
Sep 19 10:53:57.474397 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.474329    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-workload-certs\") pod \"eda19392-f366-4802-abb5-2079c705ebf0\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") "
Sep 19 10:53:57.474457 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.474425    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eda19392-f366-4802-abb5-2079c705ebf0" (UID: "eda19392-f366-4802-abb5-2079c705ebf0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:53:57.474457 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.474442    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-workload-socket\") pod \"eda19392-f366-4802-abb5-2079c705ebf0\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") "
Sep 19 10:53:57.474541 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.474497    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eda19392-f366-4802-abb5-2079c705ebf0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eda19392-f366-4802-abb5-2079c705ebf0" (UID: "eda19392-f366-4802-abb5-2079c705ebf0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:53:57.474541 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.474509    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-istio-data\") pod \"eda19392-f366-4802-abb5-2079c705ebf0\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") "
Sep 19 10:53:57.474651 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.474542    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eda19392-f366-4802-abb5-2079c705ebf0-istio-podinfo\") pod \"eda19392-f366-4802-abb5-2079c705ebf0\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") "
Sep 19 10:53:57.474651 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.474611    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eda19392-f366-4802-abb5-2079c705ebf0" (UID: "eda19392-f366-4802-abb5-2079c705ebf0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:53:57.474651 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.474630    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eda19392-f366-4802-abb5-2079c705ebf0" (UID: "eda19392-f366-4802-abb5-2079c705ebf0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:53:57.474651 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.474643    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-istio-envoy\") pod \"eda19392-f366-4802-abb5-2079c705ebf0\" (UID: \"eda19392-f366-4802-abb5-2079c705ebf0\") "
Sep 19 10:53:57.474867 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.474758    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-istio-data" (OuterVolumeSpecName: "istio-data") pod "eda19392-f366-4802-abb5-2079c705ebf0" (UID: "eda19392-f366-4802-abb5-2079c705ebf0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:53:57.475297 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.475238    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:53:57.475297 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.475272    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eda19392-f366-4802-abb5-2079c705ebf0-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:53:57.475297 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.475287    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:53:57.475521 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.475301    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:53:57.475521 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.475317    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:53:57.476451 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.476427    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eda19392-f366-4802-abb5-2079c705ebf0-kube-api-access-pdn5x" (OuterVolumeSpecName: "kube-api-access-pdn5x") pod "eda19392-f366-4802-abb5-2079c705ebf0" (UID: "eda19392-f366-4802-abb5-2079c705ebf0"). InnerVolumeSpecName "kube-api-access-pdn5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:53:57.476606 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.476590    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eda19392-f366-4802-abb5-2079c705ebf0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eda19392-f366-4802-abb5-2079c705ebf0" (UID: "eda19392-f366-4802-abb5-2079c705ebf0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 10:53:57.476770 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.476748    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eda19392-f366-4802-abb5-2079c705ebf0" (UID: "eda19392-f366-4802-abb5-2079c705ebf0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:53:57.476770 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.476758    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eda19392-f366-4802-abb5-2079c705ebf0-istio-token" (OuterVolumeSpecName: "istio-token") pod "eda19392-f366-4802-abb5-2079c705ebf0" (UID: "eda19392-f366-4802-abb5-2079c705ebf0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:53:57.576254 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.576228    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eda19392-f366-4802-abb5-2079c705ebf0-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:53:57.576254 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.576252    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eda19392-f366-4802-abb5-2079c705ebf0-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:53:57.576391 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.576266    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eda19392-f366-4802-abb5-2079c705ebf0-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:53:57.576391 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.576278    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdn5x\" (UniqueName: \"kubernetes.io/projected/eda19392-f366-4802-abb5-2079c705ebf0-kube-api-access-pdn5x\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:53:57.753807 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.753743    2576 generic.go:356] "Generic (PLEG): container finished" podID="eda19392-f366-4802-abb5-2079c705ebf0" containerID="8ec58f04240c8dd44f23a11240cffbde39fe787c618cd6bcc8936bff5b58088f" exitCode=0
Sep 19 10:53:57.753807 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.753793    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"
Sep 19 10:53:57.753967 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.753779    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg" event={"ID":"eda19392-f366-4802-abb5-2079c705ebf0","Type":"ContainerDied","Data":"8ec58f04240c8dd44f23a11240cffbde39fe787c618cd6bcc8936bff5b58088f"}
Sep 19 10:53:57.753967 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.753933    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg" event={"ID":"eda19392-f366-4802-abb5-2079c705ebf0","Type":"ContainerDied","Data":"bd98a9a842da9db769a67e34aa7f16cef93d6f8d408a9b3393718a72172ec33c"}
Sep 19 10:53:57.754119 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.753971    2576 scope.go:117] "RemoveContainer" containerID="8ec58f04240c8dd44f23a11240cffbde39fe787c618cd6bcc8936bff5b58088f"
Sep 19 10:53:57.764783 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.764766    2576 scope.go:117] "RemoveContainer" containerID="5a94ae44b5a37853af01068bd85c14826e9db55c0054686d432ab0e0929e5640"
Sep 19 10:53:57.773926 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.773910    2576 scope.go:117] "RemoveContainer" containerID="8ec58f04240c8dd44f23a11240cffbde39fe787c618cd6bcc8936bff5b58088f"
Sep 19 10:53:57.774260 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:53:57.774236    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ec58f04240c8dd44f23a11240cffbde39fe787c618cd6bcc8936bff5b58088f\": container with ID starting with 8ec58f04240c8dd44f23a11240cffbde39fe787c618cd6bcc8936bff5b58088f not found: ID does not exist" containerID="8ec58f04240c8dd44f23a11240cffbde39fe787c618cd6bcc8936bff5b58088f"
Sep 19 10:53:57.774348 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.774266    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ec58f04240c8dd44f23a11240cffbde39fe787c618cd6bcc8936bff5b58088f"} err="failed to get container status \"8ec58f04240c8dd44f23a11240cffbde39fe787c618cd6bcc8936bff5b58088f\": rpc error: code = NotFound desc = could not find container \"8ec58f04240c8dd44f23a11240cffbde39fe787c618cd6bcc8936bff5b58088f\": container with ID starting with 8ec58f04240c8dd44f23a11240cffbde39fe787c618cd6bcc8936bff5b58088f not found: ID does not exist"
Sep 19 10:53:57.774348 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.774282    2576 scope.go:117] "RemoveContainer" containerID="5a94ae44b5a37853af01068bd85c14826e9db55c0054686d432ab0e0929e5640"
Sep 19 10:53:57.774580 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:53:57.774563    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a94ae44b5a37853af01068bd85c14826e9db55c0054686d432ab0e0929e5640\": container with ID starting with 5a94ae44b5a37853af01068bd85c14826e9db55c0054686d432ab0e0929e5640 not found: ID does not exist" containerID="5a94ae44b5a37853af01068bd85c14826e9db55c0054686d432ab0e0929e5640"
Sep 19 10:53:57.774632 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.774586    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a94ae44b5a37853af01068bd85c14826e9db55c0054686d432ab0e0929e5640"} err="failed to get container status \"5a94ae44b5a37853af01068bd85c14826e9db55c0054686d432ab0e0929e5640\": rpc error: code = NotFound desc = could not find container \"5a94ae44b5a37853af01068bd85c14826e9db55c0054686d432ab0e0929e5640\": container with ID starting with 5a94ae44b5a37853af01068bd85c14826e9db55c0054686d432ab0e0929e5640 not found: ID does not exist"
Sep 19 10:53:57.775569 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.775550    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"]
Sep 19 10:53:57.786057 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:57.780931    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-18444f28-openshift-default-75f6d5ffc6-pxzjg"]
Sep 19 10:53:59.002011 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:53:59.001968    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eda19392-f366-4802-abb5-2079c705ebf0" path="/var/lib/kubelet/pods/eda19392-f366-4802-abb5-2079c705ebf0/volumes"
Sep 19 10:54:05.349366 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.349327    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"]
Sep 19 10:54:05.350085 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.350065    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e29ad04-c28b-4736-9354-0552788f872f" containerName="maas-api"
Sep 19 10:54:05.350152 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.350088    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e29ad04-c28b-4736-9354-0552788f872f" containerName="maas-api"
Sep 19 10:54:05.350152 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.350099    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eda19392-f366-4802-abb5-2079c705ebf0" containerName="istio-proxy"
Sep 19 10:54:05.350152 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.350109    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="eda19392-f366-4802-abb5-2079c705ebf0" containerName="istio-proxy"
Sep 19 10:54:05.350152 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.350137    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eda19392-f366-4802-abb5-2079c705ebf0" containerName="istio-proxy"
Sep 19 10:54:05.350152 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.350146    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="eda19392-f366-4802-abb5-2079c705ebf0" containerName="istio-proxy"
Sep 19 10:54:05.350319 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.350267    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e29ad04-c28b-4736-9354-0552788f872f" containerName="maas-api"
Sep 19 10:54:05.350319 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.350280    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="eda19392-f366-4802-abb5-2079c705ebf0" containerName="istio-proxy"
Sep 19 10:54:05.350319 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.350294    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="eda19392-f366-4802-abb5-2079c705ebf0" containerName="istio-proxy"
Sep 19 10:54:05.355601 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.355583    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.358128 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.358107    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-20c8d9-openshift-default-dockercfg-tpmd8\""
Sep 19 10:54:05.363777 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.363746    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"]
Sep 19 10:54:05.449926 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.449889    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-workload-socket\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.450115 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.449967    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-istio-envoy\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.450115 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.450022    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-istio-data\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.450115 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.450072    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/801a3275-0941-4eb7-a49c-4602ac446b96-istio-token\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.450249 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.450191    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/801a3275-0941-4eb7-a49c-4602ac446b96-istiod-ca-cert\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.450296 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.450261    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-workload-certs\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.450337 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.450291    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/801a3275-0941-4eb7-a49c-4602ac446b96-istio-podinfo\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.450373 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.450345    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrwm7\" (UniqueName: \"kubernetes.io/projected/801a3275-0941-4eb7-a49c-4602ac446b96-kube-api-access-xrwm7\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.450415 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.450395    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-credential-socket\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.551393 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.551361    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-credential-socket\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.551564 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.551421    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-workload-socket\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.551564 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.551466    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-istio-envoy\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.551564 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.551500    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-istio-data\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.551564 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.551515    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/801a3275-0941-4eb7-a49c-4602ac446b96-istio-token\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.551794 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.551585    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/801a3275-0941-4eb7-a49c-4602ac446b96-istiod-ca-cert\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.551794 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.551612    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-workload-certs\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.551794 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.551626    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/801a3275-0941-4eb7-a49c-4602ac446b96-istio-podinfo\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.551794 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.551766    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrwm7\" (UniqueName: \"kubernetes.io/projected/801a3275-0941-4eb7-a49c-4602ac446b96-kube-api-access-xrwm7\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.552018 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.551811    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-workload-socket\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.552018 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.551850    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-credential-socket\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.552018 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.551889    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-istio-data\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.552194 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.552140    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-workload-certs\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.552403 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.552269    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/801a3275-0941-4eb7-a49c-4602ac446b96-istiod-ca-cert\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.553923 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.553899    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-istio-envoy\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.554356 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.554338    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/801a3275-0941-4eb7-a49c-4602ac446b96-istio-podinfo\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.559407 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.559385    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrwm7\" (UniqueName: \"kubernetes.io/projected/801a3275-0941-4eb7-a49c-4602ac446b96-kube-api-access-xrwm7\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.559939 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.559919    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/801a3275-0941-4eb7-a49c-4602ac446b96-istio-token\") pod \"e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") " pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.669358 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.669273    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:05.946463 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.946387    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"]
Sep 19 10:54:05.950261 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:54:05.950219    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod801a3275_0941_4eb7_a49c_4602ac446b96.slice/crio-9801c8058d6c9fc525d251ac22513a6e5d50643bfd9e73f6ba349753b5ab5498 WatchSource:0}: Error finding container 9801c8058d6c9fc525d251ac22513a6e5d50643bfd9e73f6ba349753b5ab5498: Status 404 returned error can't find the container with id 9801c8058d6c9fc525d251ac22513a6e5d50643bfd9e73f6ba349753b5ab5498
Sep 19 10:54:05.952936 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.952899    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:54:05.953065 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.952963    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:54:05.953065 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:05.952991    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:54:06.807821 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:06.807775    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv" event={"ID":"801a3275-0941-4eb7-a49c-4602ac446b96","Type":"ContainerStarted","Data":"22645a86e938e2c66c85c00f49c8e8c4f934caa7ffaf7daf2fde0006f1da91a8"}
Sep 19 10:54:06.808313 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:06.807831    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv" event={"ID":"801a3275-0941-4eb7-a49c-4602ac446b96","Type":"ContainerStarted","Data":"9801c8058d6c9fc525d251ac22513a6e5d50643bfd9e73f6ba349753b5ab5498"}
Sep 19 10:54:06.827475 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:06.827430    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv" podStartSLOduration=1.82741398 podStartE2EDuration="1.82741398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:54:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:54:06.826024788 +0000 UTC m=+1056.322330879" watchObservedRunningTime="2026-09-19 10:54:06.82741398 +0000 UTC m=+1056.323720072"
Sep 19 10:54:07.670218 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:07.670189    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:07.675546 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:07.675521    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:07.814304 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:07.814273    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:07.815301 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:07.815283    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:12.115463 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.115432    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"]
Sep 19 10:54:12.119990 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.119970    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"
Sep 19 10:54:12.122253 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.122234    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-20c8d9-serving-cert\""
Sep 19 10:54:12.129191 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.129171    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"]
Sep 19 10:54:12.221411 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.221387    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcce4857-4551-4b82-8cd1-e8d2438c7699-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr\" (UID: \"fcce4857-4551-4b82-8cd1-e8d2438c7699\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"
Sep 19 10:54:12.221529 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.221426    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcce4857-4551-4b82-8cd1-e8d2438c7699-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr\" (UID: \"fcce4857-4551-4b82-8cd1-e8d2438c7699\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"
Sep 19 10:54:12.221573 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.221551    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcce4857-4551-4b82-8cd1-e8d2438c7699-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr\" (UID: \"fcce4857-4551-4b82-8cd1-e8d2438c7699\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"
Sep 19 10:54:12.221631 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.221613    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qg92\" (UniqueName: \"kubernetes.io/projected/fcce4857-4551-4b82-8cd1-e8d2438c7699-kube-api-access-6qg92\") pod \"maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr\" (UID: \"fcce4857-4551-4b82-8cd1-e8d2438c7699\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"
Sep 19 10:54:12.322844 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.322812    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qg92\" (UniqueName: \"kubernetes.io/projected/fcce4857-4551-4b82-8cd1-e8d2438c7699-kube-api-access-6qg92\") pod \"maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr\" (UID: \"fcce4857-4551-4b82-8cd1-e8d2438c7699\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"
Sep 19 10:54:12.322844 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.322849    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcce4857-4551-4b82-8cd1-e8d2438c7699-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr\" (UID: \"fcce4857-4551-4b82-8cd1-e8d2438c7699\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"
Sep 19 10:54:12.323129 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.322930    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcce4857-4551-4b82-8cd1-e8d2438c7699-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr\" (UID: \"fcce4857-4551-4b82-8cd1-e8d2438c7699\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"
Sep 19 10:54:12.323129 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.323083    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcce4857-4551-4b82-8cd1-e8d2438c7699-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr\" (UID: \"fcce4857-4551-4b82-8cd1-e8d2438c7699\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"
Sep 19 10:54:12.323596 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.323574    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcce4857-4551-4b82-8cd1-e8d2438c7699-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr\" (UID: \"fcce4857-4551-4b82-8cd1-e8d2438c7699\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"
Sep 19 10:54:12.325474 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.325449    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcce4857-4551-4b82-8cd1-e8d2438c7699-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr\" (UID: \"fcce4857-4551-4b82-8cd1-e8d2438c7699\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"
Sep 19 10:54:12.325567 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.325533    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcce4857-4551-4b82-8cd1-e8d2438c7699-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr\" (UID: \"fcce4857-4551-4b82-8cd1-e8d2438c7699\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"
Sep 19 10:54:12.330492 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.330472    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qg92\" (UniqueName: \"kubernetes.io/projected/fcce4857-4551-4b82-8cd1-e8d2438c7699-kube-api-access-6qg92\") pod \"maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr\" (UID: \"fcce4857-4551-4b82-8cd1-e8d2438c7699\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"
Sep 19 10:54:12.435136 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.435067    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"
Sep 19 10:54:12.710141 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.710033    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"]
Sep 19 10:54:12.718758 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.718736    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 10:54:12.843405 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:12.843366    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr" event={"ID":"fcce4857-4551-4b82-8cd1-e8d2438c7699","Type":"ContainerStarted","Data":"abb724b5dd3ef1be3856f1fddb6a6b73fd76cf5318d2e876a7a70b31d8493632"}
Sep 19 10:54:13.850124 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:13.850083    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr" event={"ID":"fcce4857-4551-4b82-8cd1-e8d2438c7699","Type":"ContainerStarted","Data":"278345ce84ea4ba22fc18e4d051a712290acada409198fc7607aed9d50ae5cce"}
Sep 19 10:54:13.850510 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:13.850266    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"
Sep 19 10:54:13.868388 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:13.868347    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr" podStartSLOduration=1.490535776 podStartE2EDuration="1.868334546s" totalImagesPullingTime="377.79877ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:54:13.86718007 +0000 UTC m=+1063.363486187" watchObservedRunningTime="2026-09-19 10:54:13.868334546 +0000 UTC m=+1063.364640638"
Sep 19 10:54:19.861506 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:19.861475    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"
Sep 19 10:54:27.107492 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.107456    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"]
Sep 19 10:54:27.108021 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.107713    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v" podUID="70af8e0f-55de-411e-80a9-dd921e23d99c" containerName="maas-api" containerID="cri-o://466c9a137574a691dc7b14528ef0eee58684cd1f9da66d170d769720f711dfe9" gracePeriod=30
Sep 19 10:54:27.391506 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.391477    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"
Sep 19 10:54:27.476178 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.476149    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70af8e0f-55de-411e-80a9-dd921e23d99c-metrics-certs\") pod \"70af8e0f-55de-411e-80a9-dd921e23d99c\" (UID: \"70af8e0f-55de-411e-80a9-dd921e23d99c\") "
Sep 19 10:54:27.476358 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.476191    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70af8e0f-55de-411e-80a9-dd921e23d99c-maas-api-tls\") pod \"70af8e0f-55de-411e-80a9-dd921e23d99c\" (UID: \"70af8e0f-55de-411e-80a9-dd921e23d99c\") "
Sep 19 10:54:27.476358 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.476284    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqbhq\" (UniqueName: \"kubernetes.io/projected/70af8e0f-55de-411e-80a9-dd921e23d99c-kube-api-access-jqbhq\") pod \"70af8e0f-55de-411e-80a9-dd921e23d99c\" (UID: \"70af8e0f-55de-411e-80a9-dd921e23d99c\") "
Sep 19 10:54:27.476358 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.476352    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70af8e0f-55de-411e-80a9-dd921e23d99c-openshift-service-ca\") pod \"70af8e0f-55de-411e-80a9-dd921e23d99c\" (UID: \"70af8e0f-55de-411e-80a9-dd921e23d99c\") "
Sep 19 10:54:27.476774 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.476743    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70af8e0f-55de-411e-80a9-dd921e23d99c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "70af8e0f-55de-411e-80a9-dd921e23d99c" (UID: "70af8e0f-55de-411e-80a9-dd921e23d99c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:54:27.478476 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.478453    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70af8e0f-55de-411e-80a9-dd921e23d99c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "70af8e0f-55de-411e-80a9-dd921e23d99c" (UID: "70af8e0f-55de-411e-80a9-dd921e23d99c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:54:27.478604 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.478561    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70af8e0f-55de-411e-80a9-dd921e23d99c-kube-api-access-jqbhq" (OuterVolumeSpecName: "kube-api-access-jqbhq") pod "70af8e0f-55de-411e-80a9-dd921e23d99c" (UID: "70af8e0f-55de-411e-80a9-dd921e23d99c"). InnerVolumeSpecName "kube-api-access-jqbhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:54:27.478767 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.478751    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70af8e0f-55de-411e-80a9-dd921e23d99c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "70af8e0f-55de-411e-80a9-dd921e23d99c" (UID: "70af8e0f-55de-411e-80a9-dd921e23d99c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:54:27.578075 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.578020    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqbhq\" (UniqueName: \"kubernetes.io/projected/70af8e0f-55de-411e-80a9-dd921e23d99c-kube-api-access-jqbhq\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:54:27.578075 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.578065    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70af8e0f-55de-411e-80a9-dd921e23d99c-openshift-service-ca\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:54:27.578075 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.578076    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70af8e0f-55de-411e-80a9-dd921e23d99c-metrics-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:54:27.578075 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.578085    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70af8e0f-55de-411e-80a9-dd921e23d99c-maas-api-tls\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:54:27.934424 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.934392    2576 generic.go:356] "Generic (PLEG): container finished" podID="70af8e0f-55de-411e-80a9-dd921e23d99c" containerID="466c9a137574a691dc7b14528ef0eee58684cd1f9da66d170d769720f711dfe9" exitCode=0
Sep 19 10:54:27.934587 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.934439    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"
Sep 19 10:54:27.934587 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.934480    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v" event={"ID":"70af8e0f-55de-411e-80a9-dd921e23d99c","Type":"ContainerDied","Data":"466c9a137574a691dc7b14528ef0eee58684cd1f9da66d170d769720f711dfe9"}
Sep 19 10:54:27.934587 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.934533    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v" event={"ID":"70af8e0f-55de-411e-80a9-dd921e23d99c","Type":"ContainerDied","Data":"b3006c2dea3c27bdaf6201c24298ae3247cbc708ecfa3e45f41d1a5bda5befc3"}
Sep 19 10:54:27.934587 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.934559    2576 scope.go:117] "RemoveContainer" containerID="466c9a137574a691dc7b14528ef0eee58684cd1f9da66d170d769720f711dfe9"
Sep 19 10:54:27.944701 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.944686    2576 scope.go:117] "RemoveContainer" containerID="466c9a137574a691dc7b14528ef0eee58684cd1f9da66d170d769720f711dfe9"
Sep 19 10:54:27.944957 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:54:27.944941    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"466c9a137574a691dc7b14528ef0eee58684cd1f9da66d170d769720f711dfe9\": container with ID starting with 466c9a137574a691dc7b14528ef0eee58684cd1f9da66d170d769720f711dfe9 not found: ID does not exist" containerID="466c9a137574a691dc7b14528ef0eee58684cd1f9da66d170d769720f711dfe9"
Sep 19 10:54:27.945022 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.944963    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"466c9a137574a691dc7b14528ef0eee58684cd1f9da66d170d769720f711dfe9"} err="failed to get container status \"466c9a137574a691dc7b14528ef0eee58684cd1f9da66d170d769720f711dfe9\": rpc error: code = NotFound desc = could not find container \"466c9a137574a691dc7b14528ef0eee58684cd1f9da66d170d769720f711dfe9\": container with ID starting with 466c9a137574a691dc7b14528ef0eee58684cd1f9da66d170d769720f711dfe9 not found: ID does not exist"
Sep 19 10:54:27.958759 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.958724    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"]
Sep 19 10:54:27.960865 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:27.960844    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-e898f8-59d5b95679-djf8v"]
Sep 19 10:54:28.942143 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:28.942063    2576 generic.go:356] "Generic (PLEG): container finished" podID="801a3275-0941-4eb7-a49c-4602ac446b96" containerID="22645a86e938e2c66c85c00f49c8e8c4f934caa7ffaf7daf2fde0006f1da91a8" exitCode=0
Sep 19 10:54:28.942580 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:28.942136    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv" event={"ID":"801a3275-0941-4eb7-a49c-4602ac446b96","Type":"ContainerDied","Data":"22645a86e938e2c66c85c00f49c8e8c4f934caa7ffaf7daf2fde0006f1da91a8"}
Sep 19 10:54:28.942707 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:28.942686    2576 scope.go:117] "RemoveContainer" containerID="22645a86e938e2c66c85c00f49c8e8c4f934caa7ffaf7daf2fde0006f1da91a8"
Sep 19 10:54:28.943924 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:28.943894    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:54:28.944035 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:28.943977    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:54:28.944035 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:28.944024    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:54:29.002434 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:29.002408    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70af8e0f-55de-411e-80a9-dd921e23d99c" path="/var/lib/kubelet/pods/70af8e0f-55de-411e-80a9-dd921e23d99c/volumes"
Sep 19 10:54:29.954994 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:29.954946    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv" event={"ID":"801a3275-0941-4eb7-a49c-4602ac446b96","Type":"ContainerStarted","Data":"e60cc43470331a049058f83c27ebdfc9443cb2bd0870d54cae290a3eb7a45330"}
Sep 19 10:54:30.669540 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:30.669506    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:30.669745 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:30.669667    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 19 10:54:30.669824 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:30.669734    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv" podUID="801a3275-0941-4eb7-a49c-4602ac446b96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 19 10:54:31.670730 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:31.670696    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 19 10:54:31.671176 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:31.670762    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv" podUID="801a3275-0941-4eb7-a49c-4602ac446b96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 19 10:54:32.669673 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.669638    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 19 10:54:32.669873 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.669698    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv" podUID="801a3275-0941-4eb7-a49c-4602ac446b96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 19 10:54:32.718653 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.718622    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"]
Sep 19 10:54:32.719221 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.719202    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70af8e0f-55de-411e-80a9-dd921e23d99c" containerName="maas-api"
Sep 19 10:54:32.719221 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.719221    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="70af8e0f-55de-411e-80a9-dd921e23d99c" containerName="maas-api"
Sep 19 10:54:32.719372 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.719328    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="70af8e0f-55de-411e-80a9-dd921e23d99c" containerName="maas-api"
Sep 19 10:54:32.722192 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.722172    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"
Sep 19 10:54:32.725201 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.725174    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-8ddce5-serving-cert\""
Sep 19 10:54:32.748691 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.747751    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"]
Sep 19 10:54:32.846695 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.837665    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4849b40d-7bd8-41e8-a6dd-19f6c749904d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg\" (UID: \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"
Sep 19 10:54:32.846695 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.837797    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmqqc\" (UniqueName: \"kubernetes.io/projected/4849b40d-7bd8-41e8-a6dd-19f6c749904d-kube-api-access-zmqqc\") pod \"maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg\" (UID: \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"
Sep 19 10:54:32.846695 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.837846    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4849b40d-7bd8-41e8-a6dd-19f6c749904d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg\" (UID: \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"
Sep 19 10:54:32.846695 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.837883    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4849b40d-7bd8-41e8-a6dd-19f6c749904d-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg\" (UID: \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"
Sep 19 10:54:32.939415 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.939320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4849b40d-7bd8-41e8-a6dd-19f6c749904d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg\" (UID: \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"
Sep 19 10:54:32.939585 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.939455    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmqqc\" (UniqueName: \"kubernetes.io/projected/4849b40d-7bd8-41e8-a6dd-19f6c749904d-kube-api-access-zmqqc\") pod \"maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg\" (UID: \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"
Sep 19 10:54:32.939585 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.939504    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4849b40d-7bd8-41e8-a6dd-19f6c749904d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg\" (UID: \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"
Sep 19 10:54:32.939585 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.939544    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4849b40d-7bd8-41e8-a6dd-19f6c749904d-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg\" (UID: \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"
Sep 19 10:54:32.950068 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:54:32.940183    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-serving-cert: secret "maas-api-e2e-shared-b-w2-8ddce5-serving-cert" not found
Sep 19 10:54:32.950068 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:54:32.940265    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4849b40d-7bd8-41e8-a6dd-19f6c749904d-maas-api-tls podName:4849b40d-7bd8-41e8-a6dd-19f6c749904d nodeName:}" failed. No retries permitted until 2026-09-19 10:54:33.440242968 +0000 UTC m=+1082.936549044 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4849b40d-7bd8-41e8-a6dd-19f6c749904d-maas-api-tls") pod "maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg" (UID: "4849b40d-7bd8-41e8-a6dd-19f6c749904d") : secret "maas-api-e2e-shared-b-w2-8ddce5-serving-cert" not found
Sep 19 10:54:32.950068 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.940934    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4849b40d-7bd8-41e8-a6dd-19f6c749904d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg\" (UID: \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"
Sep 19 10:54:32.950564 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.950516    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4849b40d-7bd8-41e8-a6dd-19f6c749904d-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg\" (UID: \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"
Sep 19 10:54:32.950711 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:32.950684    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmqqc\" (UniqueName: \"kubernetes.io/projected/4849b40d-7bd8-41e8-a6dd-19f6c749904d-kube-api-access-zmqqc\") pod \"maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg\" (UID: \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"
Sep 19 10:54:33.451794 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:33.451752    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4849b40d-7bd8-41e8-a6dd-19f6c749904d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg\" (UID: \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"
Sep 19 10:54:33.455618 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:33.455585    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4849b40d-7bd8-41e8-a6dd-19f6c749904d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg\" (UID: \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"
Sep 19 10:54:33.649932 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:33.649892    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"
Sep 19 10:54:33.670213 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:33.670171    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 19 10:54:33.670326 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:33.670250    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv" podUID="801a3275-0941-4eb7-a49c-4602ac446b96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 19 10:54:33.932480 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:33.932452    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"]
Sep 19 10:54:33.935734 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:54:33.935690    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4849b40d_7bd8_41e8_a6dd_19f6c749904d.slice/crio-0b0f921421efff32c01b8c849284a34fa8e743b90d3feb379bcbef8431a6021a WatchSource:0}: Error finding container 0b0f921421efff32c01b8c849284a34fa8e743b90d3feb379bcbef8431a6021a: Status 404 returned error can't find the container with id 0b0f921421efff32c01b8c849284a34fa8e743b90d3feb379bcbef8431a6021a
Sep 19 10:54:33.981303 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:33.981230    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg" event={"ID":"4849b40d-7bd8-41e8-a6dd-19f6c749904d","Type":"ContainerStarted","Data":"0b0f921421efff32c01b8c849284a34fa8e743b90d3feb379bcbef8431a6021a"}
Sep 19 10:54:34.670269 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:34.670236    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 19 10:54:34.670438 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:34.670291    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv" podUID="801a3275-0941-4eb7-a49c-4602ac446b96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 19 10:54:34.988973 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:34.988881    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg" event={"ID":"4849b40d-7bd8-41e8-a6dd-19f6c749904d","Type":"ContainerStarted","Data":"3f34870d0eacaef04906e48081f33af7c1887f50e760b5f4d6c96762fa571248"}
Sep 19 10:54:34.989394 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:34.988983    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"
Sep 19 10:54:35.012074 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:35.011990    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg" podStartSLOduration=2.733796611 podStartE2EDuration="3.011967751s" totalImagesPullingTime="278.17114ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:54:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:54:35.009766592 +0000 UTC m=+1084.506072686" watchObservedRunningTime="2026-09-19 10:54:35.011967751 +0000 UTC m=+1084.508273845"
Sep 19 10:54:35.674373 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:35.674346    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:35.674740 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:35.674722    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:35.675375 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:35.675359    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 10:54:41.002116 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:41.002090    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"
Sep 19 10:54:57.836000 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:57.835962    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7968-2cb36f"]
Sep 19 10:54:57.839765 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:57.839745    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7968-2cb36f"
Sep 19 10:54:57.842534 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:57.842335    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-m27fx\""
Sep 19 10:54:57.844631 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:57.844608    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7968-2cb36f"]
Sep 19 10:54:57.891112 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:57.891078    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwssn\" (UniqueName: \"kubernetes.io/projected/7bf8cdea-2d08-4311-ac3f-9d9d8a97f643-kube-api-access-bwssn\") pod \"test-curl-7968-2cb36f\" (UID: \"7bf8cdea-2d08-4311-ac3f-9d9d8a97f643\") " pod="opendatahub/test-curl-7968-2cb36f"
Sep 19 10:54:57.991687 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:57.991647    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwssn\" (UniqueName: \"kubernetes.io/projected/7bf8cdea-2d08-4311-ac3f-9d9d8a97f643-kube-api-access-bwssn\") pod \"test-curl-7968-2cb36f\" (UID: \"7bf8cdea-2d08-4311-ac3f-9d9d8a97f643\") " pod="opendatahub/test-curl-7968-2cb36f"
Sep 19 10:54:57.999529 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:57.999502    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwssn\" (UniqueName: \"kubernetes.io/projected/7bf8cdea-2d08-4311-ac3f-9d9d8a97f643-kube-api-access-bwssn\") pod \"test-curl-7968-2cb36f\" (UID: \"7bf8cdea-2d08-4311-ac3f-9d9d8a97f643\") " pod="opendatahub/test-curl-7968-2cb36f"
Sep 19 10:54:58.151428 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:58.151353    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7968-2cb36f"
Sep 19 10:54:58.421265 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:58.421236    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7968-2cb36f"]
Sep 19 10:54:58.422640 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:54:58.422613    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7bf8cdea_2d08_4311_ac3f_9d9d8a97f643.slice/crio-c84bbef412ec8626a951632eedcc482b8b67151d17550e3e079c8e74dcf655a1 WatchSource:0}: Error finding container c84bbef412ec8626a951632eedcc482b8b67151d17550e3e079c8e74dcf655a1: Status 404 returned error can't find the container with id c84bbef412ec8626a951632eedcc482b8b67151d17550e3e079c8e74dcf655a1
Sep 19 10:54:59.128742 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:54:59.128699    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7968-2cb36f" event={"ID":"7bf8cdea-2d08-4311-ac3f-9d9d8a97f643","Type":"ContainerStarted","Data":"c84bbef412ec8626a951632eedcc482b8b67151d17550e3e079c8e74dcf655a1"}
Sep 19 10:55:01.143852 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:01.143816    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7968-2cb36f" event={"ID":"7bf8cdea-2d08-4311-ac3f-9d9d8a97f643","Type":"ContainerStarted","Data":"ef818d53a7570d28e0f67e38f65b5a37069d8f9a5330dbf97936d8eaa06889be"}
Sep 19 10:55:01.156518 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:01.156471    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7968-2cb36f" podStartSLOduration=2.152261041 podStartE2EDuration="4.156454831s" totalImagesPullingTime="2.00419379s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:54:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:55:01.155932485 +0000 UTC m=+1110.652238577" watchObservedRunningTime="2026-09-19 10:55:01.156454831 +0000 UTC m=+1110.652760923"
Sep 19 10:55:01.722862 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:01.722829    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7968-2cb36f"]
Sep 19 10:55:01.726261 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:01.726235    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7968-2cb36f"]
Sep 19 10:55:01.838639 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:01.838613    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7968-aa5d20"]
Sep 19 10:55:01.839100 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:01.839086    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bf8cdea-2d08-4311-ac3f-9d9d8a97f643" containerName="test-curl-7968-2cb36f"
Sep 19 10:55:01.839153 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:01.839103    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bf8cdea-2d08-4311-ac3f-9d9d8a97f643" containerName="test-curl-7968-2cb36f"
Sep 19 10:55:01.839196 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:01.839187    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bf8cdea-2d08-4311-ac3f-9d9d8a97f643" containerName="test-curl-7968-2cb36f"
Sep 19 10:55:01.842881 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:01.842861    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7968-aa5d20"
Sep 19 10:55:01.845449 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:01.845426    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7968-aa5d20"]
Sep 19 10:55:01.928028 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:01.927999    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2grkr\" (UniqueName: \"kubernetes.io/projected/6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43-kube-api-access-2grkr\") pod \"test-curl-7968-aa5d20\" (UID: \"6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43\") " pod="opendatahub/test-curl-7968-aa5d20"
Sep 19 10:55:02.028530 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:02.028505    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2grkr\" (UniqueName: \"kubernetes.io/projected/6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43-kube-api-access-2grkr\") pod \"test-curl-7968-aa5d20\" (UID: \"6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43\") " pod="opendatahub/test-curl-7968-aa5d20"
Sep 19 10:55:02.049851 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:02.049817    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2grkr\" (UniqueName: \"kubernetes.io/projected/6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43-kube-api-access-2grkr\") pod \"test-curl-7968-aa5d20\" (UID: \"6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43\") " pod="opendatahub/test-curl-7968-aa5d20"
Sep 19 10:55:02.149938 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:55:02.149867    2576 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7968-2cb36f" container="test-curl-7968-2cb36f"
Sep 19 10:55:02.156011 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:02.155990    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7968-aa5d20"
Sep 19 10:55:02.426559 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:02.426535    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7968-aa5d20"]
Sep 19 10:55:02.428318 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:55:02.428292    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6bf2f9ee_ec6e_44d8_a5b0_fbb83e109f43.slice/crio-92459d6583f8bb48d08a5b6695b4aac5957229d4d0eda98944a84d2f2c5aabc0 WatchSource:0}: Error finding container 92459d6583f8bb48d08a5b6695b4aac5957229d4d0eda98944a84d2f2c5aabc0: Status 404 returned error can't find the container with id 92459d6583f8bb48d08a5b6695b4aac5957229d4d0eda98944a84d2f2c5aabc0
Sep 19 10:55:03.156982 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:03.156952    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7968-aa5d20" event={"ID":"6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43","Type":"ContainerStarted","Data":"5d42fa2306f05b511e69ef3669001d5d92daefeca8c575f5dcb931e640930f9d"}
Sep 19 10:55:03.157450 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:03.156992    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7968-aa5d20" event={"ID":"6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43","Type":"ContainerStarted","Data":"92459d6583f8bb48d08a5b6695b4aac5957229d4d0eda98944a84d2f2c5aabc0"}
Sep 19 10:55:03.157450 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:03.157077    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7968-2cb36f" podUID="7bf8cdea-2d08-4311-ac3f-9d9d8a97f643" containerName="test-curl-7968-2cb36f" containerID="cri-o://ef818d53a7570d28e0f67e38f65b5a37069d8f9a5330dbf97936d8eaa06889be" gracePeriod=2
Sep 19 10:55:03.172189 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:03.172146    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7968-aa5d20" podStartSLOduration=2.172132193 podStartE2EDuration="2.172132193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:55:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:55:03.169767217 +0000 UTC m=+1112.666073309" watchObservedRunningTime="2026-09-19 10:55:03.172132193 +0000 UTC m=+1112.668438321"
Sep 19 10:55:03.736503 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:03.736442    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7968-aa5d20"]
Sep 19 10:55:03.739102 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:03.739074    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7968-aa5d20"]
Sep 19 10:55:03.856321 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:03.856291    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7968-79748d"]
Sep 19 10:55:03.856749 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:03.856737    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43" containerName="test-curl-7968-aa5d20"
Sep 19 10:55:03.856794 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:03.856751    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43" containerName="test-curl-7968-aa5d20"
Sep 19 10:55:03.856850 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:03.856841    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43" containerName="test-curl-7968-aa5d20"
Sep 19 10:55:03.860207 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:03.860192    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7968-79748d"
Sep 19 10:55:03.865723 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:03.865700    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7968-79748d"]
Sep 19 10:55:03.946237 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:03.946214    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5bmn\" (UniqueName: \"kubernetes.io/projected/1a8457e3-b51f-4afd-8229-2b7b3e43ed0c-kube-api-access-k5bmn\") pod \"test-curl-7968-79748d\" (UID: \"1a8457e3-b51f-4afd-8229-2b7b3e43ed0c\") " pod="opendatahub/test-curl-7968-79748d"
Sep 19 10:55:04.047751 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:04.047721    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5bmn\" (UniqueName: \"kubernetes.io/projected/1a8457e3-b51f-4afd-8229-2b7b3e43ed0c-kube-api-access-k5bmn\") pod \"test-curl-7968-79748d\" (UID: \"1a8457e3-b51f-4afd-8229-2b7b3e43ed0c\") " pod="opendatahub/test-curl-7968-79748d"
Sep 19 10:55:04.055158 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:04.055138    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5bmn\" (UniqueName: \"kubernetes.io/projected/1a8457e3-b51f-4afd-8229-2b7b3e43ed0c-kube-api-access-k5bmn\") pod \"test-curl-7968-79748d\" (UID: \"1a8457e3-b51f-4afd-8229-2b7b3e43ed0c\") " pod="opendatahub/test-curl-7968-79748d"
Sep 19 10:55:04.162525 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:55:04.162465    2576 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7968-aa5d20" container="test-curl-7968-aa5d20"
Sep 19 10:55:04.171202 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:04.171182    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7968-79748d"
Sep 19 10:55:04.440219 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:04.440194    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7968-79748d"]
Sep 19 10:55:04.442428 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:55:04.442403    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a8457e3_b51f_4afd_8229_2b7b3e43ed0c.slice/crio-6533f611664e65b25520e9b87f925da9d2ab27b70b86b252077813f722aea725 WatchSource:0}: Error finding container 6533f611664e65b25520e9b87f925da9d2ab27b70b86b252077813f722aea725: Status 404 returned error can't find the container with id 6533f611664e65b25520e9b87f925da9d2ab27b70b86b252077813f722aea725
Sep 19 10:55:05.169894 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:05.169859    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7968-79748d" event={"ID":"1a8457e3-b51f-4afd-8229-2b7b3e43ed0c","Type":"ContainerStarted","Data":"4351cf1335a6f837eff68a1f58bfa390dcfc37799cb6ce0cab9cd10a6d056c6e"}
Sep 19 10:55:05.170269 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:05.169912    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7968-79748d" event={"ID":"1a8457e3-b51f-4afd-8229-2b7b3e43ed0c","Type":"ContainerStarted","Data":"6533f611664e65b25520e9b87f925da9d2ab27b70b86b252077813f722aea725"}
Sep 19 10:55:05.170269 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:05.169970    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7968-aa5d20" podUID="6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43" containerName="test-curl-7968-aa5d20" containerID="cri-o://5d42fa2306f05b511e69ef3669001d5d92daefeca8c575f5dcb931e640930f9d" gracePeriod=2
Sep 19 10:55:05.185224 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:05.185184    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7968-79748d" podStartSLOduration=2.185168324 podStartE2EDuration="2.185168324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:55:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:55:05.182698329 +0000 UTC m=+1114.679004422" watchObservedRunningTime="2026-09-19 10:55:05.185168324 +0000 UTC m=+1114.681474416"
Sep 19 10:55:05.340379 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:05.340358    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7968-2cb36f"
Sep 19 10:55:05.461241 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:05.461175    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwssn\" (UniqueName: \"kubernetes.io/projected/7bf8cdea-2d08-4311-ac3f-9d9d8a97f643-kube-api-access-bwssn\") pod \"7bf8cdea-2d08-4311-ac3f-9d9d8a97f643\" (UID: \"7bf8cdea-2d08-4311-ac3f-9d9d8a97f643\") "
Sep 19 10:55:05.463247 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:05.463225    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bf8cdea-2d08-4311-ac3f-9d9d8a97f643-kube-api-access-bwssn" (OuterVolumeSpecName: "kube-api-access-bwssn") pod "7bf8cdea-2d08-4311-ac3f-9d9d8a97f643" (UID: "7bf8cdea-2d08-4311-ac3f-9d9d8a97f643"). InnerVolumeSpecName "kube-api-access-bwssn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:55:05.562737 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:05.562710    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwssn\" (UniqueName: \"kubernetes.io/projected/7bf8cdea-2d08-4311-ac3f-9d9d8a97f643-kube-api-access-bwssn\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:55:05.748514 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:05.748435    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7968-79748d"]
Sep 19 10:55:05.752765 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:05.752737    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7968-79748d"]
Sep 19 10:55:05.868309 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:05.868278    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7968-de5093"]
Sep 19 10:55:05.868819 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:05.868807    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a8457e3-b51f-4afd-8229-2b7b3e43ed0c" containerName="test-curl-7968-79748d"
Sep 19 10:55:05.868869 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:05.868821    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a8457e3-b51f-4afd-8229-2b7b3e43ed0c" containerName="test-curl-7968-79748d"
Sep 19 10:55:05.868921 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:05.868912    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a8457e3-b51f-4afd-8229-2b7b3e43ed0c" containerName="test-curl-7968-79748d"
Sep 19 10:55:05.872336 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:05.872320    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7968-de5093"
Sep 19 10:55:05.876770 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:05.876745    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7968-de5093"]
Sep 19 10:55:05.967791 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:05.967755    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s2dc\" (UniqueName: \"kubernetes.io/projected/218cba6e-2cad-4f20-b9b2-c92666995992-kube-api-access-6s2dc\") pod \"test-curl-7968-de5093\" (UID: \"218cba6e-2cad-4f20-b9b2-c92666995992\") " pod="opendatahub/test-curl-7968-de5093"
Sep 19 10:55:06.068907 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:06.068820    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s2dc\" (UniqueName: \"kubernetes.io/projected/218cba6e-2cad-4f20-b9b2-c92666995992-kube-api-access-6s2dc\") pod \"test-curl-7968-de5093\" (UID: \"218cba6e-2cad-4f20-b9b2-c92666995992\") " pod="opendatahub/test-curl-7968-de5093"
Sep 19 10:55:06.077066 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:06.077015    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s2dc\" (UniqueName: \"kubernetes.io/projected/218cba6e-2cad-4f20-b9b2-c92666995992-kube-api-access-6s2dc\") pod \"test-curl-7968-de5093\" (UID: \"218cba6e-2cad-4f20-b9b2-c92666995992\") " pod="opendatahub/test-curl-7968-de5093"
Sep 19 10:55:06.176486 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:06.176451    2576 generic.go:356] "Generic (PLEG): container finished" podID="7bf8cdea-2d08-4311-ac3f-9d9d8a97f643" containerID="ef818d53a7570d28e0f67e38f65b5a37069d8f9a5330dbf97936d8eaa06889be" exitCode=137
Sep 19 10:55:06.176948 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:06.176522    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7968-2cb36f"
Sep 19 10:55:06.176948 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:06.176541    2576 scope.go:117] "RemoveContainer" containerID="ef818d53a7570d28e0f67e38f65b5a37069d8f9a5330dbf97936d8eaa06889be"
Sep 19 10:55:06.176948 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:55:06.176885    2576 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7968-79748d" container="test-curl-7968-79748d"
Sep 19 10:55:06.183378 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:06.183356    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7968-de5093"
Sep 19 10:55:06.185946 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:06.185929    2576 scope.go:117] "RemoveContainer" containerID="ef818d53a7570d28e0f67e38f65b5a37069d8f9a5330dbf97936d8eaa06889be"
Sep 19 10:55:06.186238 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:55:06.186221    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef818d53a7570d28e0f67e38f65b5a37069d8f9a5330dbf97936d8eaa06889be\": container with ID starting with ef818d53a7570d28e0f67e38f65b5a37069d8f9a5330dbf97936d8eaa06889be not found: ID does not exist" containerID="ef818d53a7570d28e0f67e38f65b5a37069d8f9a5330dbf97936d8eaa06889be"
Sep 19 10:55:06.186289 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:06.186246    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef818d53a7570d28e0f67e38f65b5a37069d8f9a5330dbf97936d8eaa06889be"} err="failed to get container status \"ef818d53a7570d28e0f67e38f65b5a37069d8f9a5330dbf97936d8eaa06889be\": rpc error: code = NotFound desc = could not find container \"ef818d53a7570d28e0f67e38f65b5a37069d8f9a5330dbf97936d8eaa06889be\": container with ID starting with ef818d53a7570d28e0f67e38f65b5a37069d8f9a5330dbf97936d8eaa06889be not found: ID does not exist"
Sep 19 10:55:06.187165 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:55:06.187141    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7968-2cb36f\" is forbidden: User \"system:node:ip-10-0-142-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-165.ec2.internal' and this object" podUID="7bf8cdea-2d08-4311-ac3f-9d9d8a97f643" pod="opendatahub/test-curl-7968-2cb36f"
Sep 19 10:55:06.456119 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:06.456092    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7968-de5093"]
Sep 19 10:55:06.458316 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:55:06.458295    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod218cba6e_2cad_4f20_b9b2_c92666995992.slice/crio-ba852905bbb56a8c2809b612ad7fb7205e79b6a1054d471e34f8b2f67425e328 WatchSource:0}: Error finding container ba852905bbb56a8c2809b612ad7fb7205e79b6a1054d471e34f8b2f67425e328: Status 404 returned error can't find the container with id ba852905bbb56a8c2809b612ad7fb7205e79b6a1054d471e34f8b2f67425e328
Sep 19 10:55:07.002304 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:07.002273    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bf8cdea-2d08-4311-ac3f-9d9d8a97f643" path="/var/lib/kubelet/pods/7bf8cdea-2d08-4311-ac3f-9d9d8a97f643/volumes"
Sep 19 10:55:07.184147 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:07.184106    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7968-de5093" event={"ID":"218cba6e-2cad-4f20-b9b2-c92666995992","Type":"ContainerStarted","Data":"e97fe766a8f4672e0f3ca181f4ea71222034fbea2befbdf97336fef157f97039"}
Sep 19 10:55:07.184528 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:07.184163    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7968-de5093" event={"ID":"218cba6e-2cad-4f20-b9b2-c92666995992","Type":"ContainerStarted","Data":"ba852905bbb56a8c2809b612ad7fb7205e79b6a1054d471e34f8b2f67425e328"}
Sep 19 10:55:07.185335 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:07.185304    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7968-79748d" podUID="1a8457e3-b51f-4afd-8229-2b7b3e43ed0c" containerName="test-curl-7968-79748d" containerID="cri-o://4351cf1335a6f837eff68a1f58bfa390dcfc37799cb6ce0cab9cd10a6d056c6e" gracePeriod=2
Sep 19 10:55:07.200009 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:07.199958    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7968-de5093" podStartSLOduration=2.19994125 podStartE2EDuration="2.19994125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:55:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:55:07.19814905 +0000 UTC m=+1116.694455144" watchObservedRunningTime="2026-09-19 10:55:07.19994125 +0000 UTC m=+1116.696247341"
Sep 19 10:55:07.354239 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:07.354218    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7968-aa5d20"
Sep 19 10:55:07.485101 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:07.485069    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2grkr\" (UniqueName: \"kubernetes.io/projected/6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43-kube-api-access-2grkr\") pod \"6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43\" (UID: \"6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43\") "
Sep 19 10:55:07.487311 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:07.487279    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43-kube-api-access-2grkr" (OuterVolumeSpecName: "kube-api-access-2grkr") pod "6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43" (UID: "6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43"). InnerVolumeSpecName "kube-api-access-2grkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:55:07.586662 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:07.586629    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2grkr\" (UniqueName: \"kubernetes.io/projected/6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43-kube-api-access-2grkr\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:55:07.778843 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:07.778810    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7968-de5093"]
Sep 19 10:55:07.782367 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:07.782327    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7968-de5093"]
Sep 19 10:55:08.192651 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:08.192616    2576 generic.go:356] "Generic (PLEG): container finished" podID="6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43" containerID="5d42fa2306f05b511e69ef3669001d5d92daefeca8c575f5dcb931e640930f9d" exitCode=137
Sep 19 10:55:08.193155 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:08.192665    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7968-aa5d20"
Sep 19 10:55:08.193155 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:08.192681    2576 scope.go:117] "RemoveContainer" containerID="5d42fa2306f05b511e69ef3669001d5d92daefeca8c575f5dcb931e640930f9d"
Sep 19 10:55:08.193155 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:55:08.192930    2576 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7968-de5093" container="test-curl-7968-de5093"
Sep 19 10:55:08.202790 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:08.202774    2576 scope.go:117] "RemoveContainer" containerID="5d42fa2306f05b511e69ef3669001d5d92daefeca8c575f5dcb931e640930f9d"
Sep 19 10:55:08.203078 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:55:08.203062    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d42fa2306f05b511e69ef3669001d5d92daefeca8c575f5dcb931e640930f9d\": container with ID starting with 5d42fa2306f05b511e69ef3669001d5d92daefeca8c575f5dcb931e640930f9d not found: ID does not exist" containerID="5d42fa2306f05b511e69ef3669001d5d92daefeca8c575f5dcb931e640930f9d"
Sep 19 10:55:08.203130 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:08.203086    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d42fa2306f05b511e69ef3669001d5d92daefeca8c575f5dcb931e640930f9d"} err="failed to get container status \"5d42fa2306f05b511e69ef3669001d5d92daefeca8c575f5dcb931e640930f9d\": rpc error: code = NotFound desc = could not find container \"5d42fa2306f05b511e69ef3669001d5d92daefeca8c575f5dcb931e640930f9d\": container with ID starting with 5d42fa2306f05b511e69ef3669001d5d92daefeca8c575f5dcb931e640930f9d not found: ID does not exist"
Sep 19 10:55:08.205480 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:55:08.205458    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7968-aa5d20\" is forbidden: User \"system:node:ip-10-0-142-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-165.ec2.internal' and this object" podUID="6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43" pod="opendatahub/test-curl-7968-aa5d20"
Sep 19 10:55:09.002115 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:09.002082    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43" path="/var/lib/kubelet/pods/6bf2f9ee-ec6e-44d8-a5b0-fbb83e109f43/volumes"
Sep 19 10:55:09.199602 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:09.199573    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7968-de5093" podUID="218cba6e-2cad-4f20-b9b2-c92666995992" containerName="test-curl-7968-de5093" containerID="cri-o://e97fe766a8f4672e0f3ca181f4ea71222034fbea2befbdf97336fef157f97039" gracePeriod=2
Sep 19 10:55:09.385972 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:09.385947    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7968-79748d"
Sep 19 10:55:09.507248 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:09.507216    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5bmn\" (UniqueName: \"kubernetes.io/projected/1a8457e3-b51f-4afd-8229-2b7b3e43ed0c-kube-api-access-k5bmn\") pod \"1a8457e3-b51f-4afd-8229-2b7b3e43ed0c\" (UID: \"1a8457e3-b51f-4afd-8229-2b7b3e43ed0c\") "
Sep 19 10:55:09.509435 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:09.509413    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a8457e3-b51f-4afd-8229-2b7b3e43ed0c-kube-api-access-k5bmn" (OuterVolumeSpecName: "kube-api-access-k5bmn") pod "1a8457e3-b51f-4afd-8229-2b7b3e43ed0c" (UID: "1a8457e3-b51f-4afd-8229-2b7b3e43ed0c"). InnerVolumeSpecName "kube-api-access-k5bmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:55:09.609109 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:09.609069    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5bmn\" (UniqueName: \"kubernetes.io/projected/1a8457e3-b51f-4afd-8229-2b7b3e43ed0c-kube-api-access-k5bmn\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:55:10.151558 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:10.151526    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"]
Sep 19 10:55:10.151818 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:10.151784    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg" podUID="cadfa4e9-7d31-487d-8b57-4087c39c917a" containerName="istio-proxy" containerID="cri-o://22b423d5db5117bad10919b25546cc93b4e7ce2c0e7cf313ecd55afaaa055b38" gracePeriod=30
Sep 19 10:55:10.209383 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:10.209345    2576 generic.go:356] "Generic (PLEG): container finished" podID="1a8457e3-b51f-4afd-8229-2b7b3e43ed0c" containerID="4351cf1335a6f837eff68a1f58bfa390dcfc37799cb6ce0cab9cd10a6d056c6e" exitCode=137
Sep 19 10:55:10.209809 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:10.209390    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7968-79748d"
Sep 19 10:55:10.209809 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:10.209461    2576 scope.go:117] "RemoveContainer" containerID="4351cf1335a6f837eff68a1f58bfa390dcfc37799cb6ce0cab9cd10a6d056c6e"
Sep 19 10:55:10.219976 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:10.219953    2576 scope.go:117] "RemoveContainer" containerID="4351cf1335a6f837eff68a1f58bfa390dcfc37799cb6ce0cab9cd10a6d056c6e"
Sep 19 10:55:10.220267 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:55:10.220247    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4351cf1335a6f837eff68a1f58bfa390dcfc37799cb6ce0cab9cd10a6d056c6e\": container with ID starting with 4351cf1335a6f837eff68a1f58bfa390dcfc37799cb6ce0cab9cd10a6d056c6e not found: ID does not exist" containerID="4351cf1335a6f837eff68a1f58bfa390dcfc37799cb6ce0cab9cd10a6d056c6e"
Sep 19 10:55:10.220355 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:10.220284    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4351cf1335a6f837eff68a1f58bfa390dcfc37799cb6ce0cab9cd10a6d056c6e"} err="failed to get container status \"4351cf1335a6f837eff68a1f58bfa390dcfc37799cb6ce0cab9cd10a6d056c6e\": rpc error: code = NotFound desc = could not find container \"4351cf1335a6f837eff68a1f58bfa390dcfc37799cb6ce0cab9cd10a6d056c6e\": container with ID starting with 4351cf1335a6f837eff68a1f58bfa390dcfc37799cb6ce0cab9cd10a6d056c6e not found: ID does not exist"
Sep 19 10:55:10.220355 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:55:10.220244    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7968-79748d\" is forbidden: User \"system:node:ip-10-0-142-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-165.ec2.internal' and this object" podUID="1a8457e3-b51f-4afd-8229-2b7b3e43ed0c" pod="opendatahub/test-curl-7968-79748d"
Sep 19 10:55:10.871862 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:10.871831    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"]
Sep 19 10:55:10.872402 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:10.872383    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="218cba6e-2cad-4f20-b9b2-c92666995992" containerName="test-curl-7968-de5093"
Sep 19 10:55:10.872402 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:10.872401    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="218cba6e-2cad-4f20-b9b2-c92666995992" containerName="test-curl-7968-de5093"
Sep 19 10:55:10.872564 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:10.872517    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="218cba6e-2cad-4f20-b9b2-c92666995992" containerName="test-curl-7968-de5093"
Sep 19 10:55:10.877224 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:10.877205    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:10.880748 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:55:10.880713    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7968-79748d\" is forbidden: User \"system:node:ip-10-0-142-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-165.ec2.internal' and this object" podUID="1a8457e3-b51f-4afd-8229-2b7b3e43ed0c" pod="opendatahub/test-curl-7968-79748d"
Sep 19 10:55:10.880850 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:10.880814    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-5fd2ea-openshift-default-dockercfg-ckcss\""
Sep 19 10:55:10.888229 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:10.888204    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"]
Sep 19 10:55:11.001997 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:55:11.001960    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7968-79748d\" is forbidden: User \"system:node:ip-10-0-142-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-165.ec2.internal' and this object" podUID="1a8457e3-b51f-4afd-8229-2b7b3e43ed0c" pod="opendatahub/test-curl-7968-79748d"
Sep 19 10:55:11.003456 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.003435    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a8457e3-b51f-4afd-8229-2b7b3e43ed0c" path="/var/lib/kubelet/pods/1a8457e3-b51f-4afd-8229-2b7b3e43ed0c/volumes"
Sep 19 10:55:11.022219 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.022191    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-data\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.022331 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.022254    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-credential-socket\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.022383 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.022343    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-token\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.022383 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.022372    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b1d66fc-64ad-439f-8618-14ce14194ea8-istiod-ca-cert\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.022453 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.022400    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-podinfo\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.022453 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.022418    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsz27\" (UniqueName: \"kubernetes.io/projected/8b1d66fc-64ad-439f-8618-14ce14194ea8-kube-api-access-rsz27\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.022516 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.022452    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-envoy\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.022516 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.022469    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-workload-socket\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.022516 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.022497    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-workload-certs\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.123482 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.123394    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-data\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.123482 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.123456    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-credential-socket\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.123697 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.123520    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-token\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.123697 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.123550    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b1d66fc-64ad-439f-8618-14ce14194ea8-istiod-ca-cert\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.123697 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.123584    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-podinfo\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.123697 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.123608    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsz27\" (UniqueName: \"kubernetes.io/projected/8b1d66fc-64ad-439f-8618-14ce14194ea8-kube-api-access-rsz27\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.123697 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.123637    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-envoy\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.123697 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.123663    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-workload-socket\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.124020 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.123696    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-workload-certs\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.124020 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.123830    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-data\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.124020 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.123844    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-credential-socket\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.124210 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.124140    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-workload-certs\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.124210 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.124152    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-workload-socket\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.124326 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.124303    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b1d66fc-64ad-439f-8618-14ce14194ea8-istiod-ca-cert\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.126129 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.126104    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-envoy\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.126301 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.126285    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-podinfo\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.147196 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.147169    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-token\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.147294 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.147281    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsz27\" (UniqueName: \"kubernetes.io/projected/8b1d66fc-64ad-439f-8618-14ce14194ea8-kube-api-access-rsz27\") pod \"e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") " pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.190715 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.190683    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:11.379306 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.379248    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7968-de5093"
Sep 19 10:55:11.426193 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.426165    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6s2dc\" (UniqueName: \"kubernetes.io/projected/218cba6e-2cad-4f20-b9b2-c92666995992-kube-api-access-6s2dc\") pod \"218cba6e-2cad-4f20-b9b2-c92666995992\" (UID: \"218cba6e-2cad-4f20-b9b2-c92666995992\") "
Sep 19 10:55:11.428411 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.428387    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/218cba6e-2cad-4f20-b9b2-c92666995992-kube-api-access-6s2dc" (OuterVolumeSpecName: "kube-api-access-6s2dc") pod "218cba6e-2cad-4f20-b9b2-c92666995992" (UID: "218cba6e-2cad-4f20-b9b2-c92666995992"). InnerVolumeSpecName "kube-api-access-6s2dc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:55:11.493160 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.493127    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"]
Sep 19 10:55:11.494659 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:55:11.494637    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b1d66fc_64ad_439f_8618_14ce14194ea8.slice/crio-e4e0933ce36314e2baccc99f70e02f68032cfee3ae9013a1a3d2ba1922e832ef WatchSource:0}: Error finding container e4e0933ce36314e2baccc99f70e02f68032cfee3ae9013a1a3d2ba1922e832ef: Status 404 returned error can't find the container with id e4e0933ce36314e2baccc99f70e02f68032cfee3ae9013a1a3d2ba1922e832ef
Sep 19 10:55:11.497178 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.497146    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:55:11.497245 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.497210    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:55:11.497245 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.497239    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:55:11.527928 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:11.527883    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6s2dc\" (UniqueName: \"kubernetes.io/projected/218cba6e-2cad-4f20-b9b2-c92666995992-kube-api-access-6s2dc\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:55:12.232084 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:12.232021    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv" event={"ID":"8b1d66fc-64ad-439f-8618-14ce14194ea8","Type":"ContainerStarted","Data":"45dbaaada28ad80e662dcecaddd227df952d25cfd3f61c99e250970bc655cfe1"}
Sep 19 10:55:12.232258 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:12.232116    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv" event={"ID":"8b1d66fc-64ad-439f-8618-14ce14194ea8","Type":"ContainerStarted","Data":"e4e0933ce36314e2baccc99f70e02f68032cfee3ae9013a1a3d2ba1922e832ef"}
Sep 19 10:55:12.233422 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:12.233396    2576 generic.go:356] "Generic (PLEG): container finished" podID="218cba6e-2cad-4f20-b9b2-c92666995992" containerID="e97fe766a8f4672e0f3ca181f4ea71222034fbea2befbdf97336fef157f97039" exitCode=137
Sep 19 10:55:12.233551 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:12.233430    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7968-de5093"
Sep 19 10:55:12.233551 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:12.233498    2576 scope.go:117] "RemoveContainer" containerID="e97fe766a8f4672e0f3ca181f4ea71222034fbea2befbdf97336fef157f97039"
Sep 19 10:55:12.242986 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:12.242967    2576 scope.go:117] "RemoveContainer" containerID="e97fe766a8f4672e0f3ca181f4ea71222034fbea2befbdf97336fef157f97039"
Sep 19 10:55:12.243285 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:55:12.243264    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e97fe766a8f4672e0f3ca181f4ea71222034fbea2befbdf97336fef157f97039\": container with ID starting with e97fe766a8f4672e0f3ca181f4ea71222034fbea2befbdf97336fef157f97039 not found: ID does not exist" containerID="e97fe766a8f4672e0f3ca181f4ea71222034fbea2befbdf97336fef157f97039"
Sep 19 10:55:12.243362 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:12.243293    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e97fe766a8f4672e0f3ca181f4ea71222034fbea2befbdf97336fef157f97039"} err="failed to get container status \"e97fe766a8f4672e0f3ca181f4ea71222034fbea2befbdf97336fef157f97039\": rpc error: code = NotFound desc = could not find container \"e97fe766a8f4672e0f3ca181f4ea71222034fbea2befbdf97336fef157f97039\": container with ID starting with e97fe766a8f4672e0f3ca181f4ea71222034fbea2befbdf97336fef157f97039 not found: ID does not exist"
Sep 19 10:55:12.252936 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:12.252893    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv" podStartSLOduration=2.25287888 podStartE2EDuration="2.25287888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:55:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:55:12.251594138 +0000 UTC m=+1121.747900254" watchObservedRunningTime="2026-09-19 10:55:12.25287888 +0000 UTC m=+1121.749184972"
Sep 19 10:55:12.253637 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:55:12.253616    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7968-de5093\" is forbidden: User \"system:node:ip-10-0-142-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-165.ec2.internal' and this object" podUID="218cba6e-2cad-4f20-b9b2-c92666995992" pod="opendatahub/test-curl-7968-de5093"
Sep 19 10:55:13.002725 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:13.002689    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="218cba6e-2cad-4f20-b9b2-c92666995992" path="/var/lib/kubelet/pods/218cba6e-2cad-4f20-b9b2-c92666995992/volumes"
Sep 19 10:55:13.190993 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:13.190961    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:13.196342 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:13.196320    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:13.241026 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:13.240993    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:13.241917 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:13.241897    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:55:15.424529 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.424465    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:55:15.571890 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.571855    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-workload-certs\") pod \"cadfa4e9-7d31-487d-8b57-4087c39c917a\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") "
Sep 19 10:55:15.571890 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.571894    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-data\") pod \"cadfa4e9-7d31-487d-8b57-4087c39c917a\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") "
Sep 19 10:55:15.572173 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.571929    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-podinfo\") pod \"cadfa4e9-7d31-487d-8b57-4087c39c917a\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") "
Sep 19 10:55:15.572173 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.571980    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-workload-socket\") pod \"cadfa4e9-7d31-487d-8b57-4087c39c917a\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") "
Sep 19 10:55:15.572173 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.571998    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjdbn\" (UniqueName: \"kubernetes.io/projected/cadfa4e9-7d31-487d-8b57-4087c39c917a-kube-api-access-bjdbn\") pod \"cadfa4e9-7d31-487d-8b57-4087c39c917a\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") "
Sep 19 10:55:15.572173 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.572016    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-envoy\") pod \"cadfa4e9-7d31-487d-8b57-4087c39c917a\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") "
Sep 19 10:55:15.572380 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.572185    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-credential-socket\") pod \"cadfa4e9-7d31-487d-8b57-4087c39c917a\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") "
Sep 19 10:55:15.572380 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.572255    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-token\") pod \"cadfa4e9-7d31-487d-8b57-4087c39c917a\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") "
Sep 19 10:55:15.572380 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.572258    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-data" (OuterVolumeSpecName: "istio-data") pod "cadfa4e9-7d31-487d-8b57-4087c39c917a" (UID: "cadfa4e9-7d31-487d-8b57-4087c39c917a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:55:15.572380 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.572275    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cadfa4e9-7d31-487d-8b57-4087c39c917a" (UID: "cadfa4e9-7d31-487d-8b57-4087c39c917a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:55:15.572380 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.572286    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cadfa4e9-7d31-487d-8b57-4087c39c917a-istiod-ca-cert\") pod \"cadfa4e9-7d31-487d-8b57-4087c39c917a\" (UID: \"cadfa4e9-7d31-487d-8b57-4087c39c917a\") "
Sep 19 10:55:15.572599 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.572393    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cadfa4e9-7d31-487d-8b57-4087c39c917a" (UID: "cadfa4e9-7d31-487d-8b57-4087c39c917a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:55:15.572599 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.572497    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cadfa4e9-7d31-487d-8b57-4087c39c917a" (UID: "cadfa4e9-7d31-487d-8b57-4087c39c917a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:55:15.572599 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.572587    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cadfa4e9-7d31-487d-8b57-4087c39c917a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cadfa4e9-7d31-487d-8b57-4087c39c917a" (UID: "cadfa4e9-7d31-487d-8b57-4087c39c917a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:55:15.572829 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.572814    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:55:15.572867 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.572832    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:55:15.572867 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.572840    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:55:15.572867 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.572848    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:55:15.572867 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.572857    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cadfa4e9-7d31-487d-8b57-4087c39c917a-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:55:15.574384 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.574355    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cadfa4e9-7d31-487d-8b57-4087c39c917a-kube-api-access-bjdbn" (OuterVolumeSpecName: "kube-api-access-bjdbn") pod "cadfa4e9-7d31-487d-8b57-4087c39c917a" (UID: "cadfa4e9-7d31-487d-8b57-4087c39c917a"). InnerVolumeSpecName "kube-api-access-bjdbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:55:15.574888 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.574856    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cadfa4e9-7d31-487d-8b57-4087c39c917a" (UID: "cadfa4e9-7d31-487d-8b57-4087c39c917a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:55:15.574996 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.574906    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cadfa4e9-7d31-487d-8b57-4087c39c917a" (UID: "cadfa4e9-7d31-487d-8b57-4087c39c917a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 10:55:15.574996 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.574957    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-token" (OuterVolumeSpecName: "istio-token") pod "cadfa4e9-7d31-487d-8b57-4087c39c917a" (UID: "cadfa4e9-7d31-487d-8b57-4087c39c917a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:55:15.673517 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.673481    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:55:15.673517 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.673512    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjdbn\" (UniqueName: \"kubernetes.io/projected/cadfa4e9-7d31-487d-8b57-4087c39c917a-kube-api-access-bjdbn\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:55:15.673517 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.673525    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:55:15.673734 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:15.673537    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cadfa4e9-7d31-487d-8b57-4087c39c917a-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:55:16.263125 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:16.263017    2576 generic.go:356] "Generic (PLEG): container finished" podID="cadfa4e9-7d31-487d-8b57-4087c39c917a" containerID="22b423d5db5117bad10919b25546cc93b4e7ce2c0e7cf313ecd55afaaa055b38" exitCode=0
Sep 19 10:55:16.263330 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:16.263112    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg" event={"ID":"cadfa4e9-7d31-487d-8b57-4087c39c917a","Type":"ContainerDied","Data":"22b423d5db5117bad10919b25546cc93b4e7ce2c0e7cf313ecd55afaaa055b38"}
Sep 19 10:55:16.263330 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:16.263210    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg" event={"ID":"cadfa4e9-7d31-487d-8b57-4087c39c917a","Type":"ContainerDied","Data":"e6c08216e36ab2b04ba1fa9d59f77a20f623054ddd61b74c6cde6ed52cee0b9c"}
Sep 19 10:55:16.263330 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:16.263257    2576 scope.go:117] "RemoveContainer" containerID="22b423d5db5117bad10919b25546cc93b4e7ce2c0e7cf313ecd55afaaa055b38"
Sep 19 10:55:16.263622 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:16.263599    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"
Sep 19 10:55:16.274306 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:16.274207    2576 scope.go:117] "RemoveContainer" containerID="5e286408bca4e0bc26339e4c017f8e622d152369a5c67a329a48699d5cc29f29"
Sep 19 10:55:16.282942 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:16.282923    2576 scope.go:117] "RemoveContainer" containerID="22b423d5db5117bad10919b25546cc93b4e7ce2c0e7cf313ecd55afaaa055b38"
Sep 19 10:55:16.283218 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:55:16.283194    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22b423d5db5117bad10919b25546cc93b4e7ce2c0e7cf313ecd55afaaa055b38\": container with ID starting with 22b423d5db5117bad10919b25546cc93b4e7ce2c0e7cf313ecd55afaaa055b38 not found: ID does not exist" containerID="22b423d5db5117bad10919b25546cc93b4e7ce2c0e7cf313ecd55afaaa055b38"
Sep 19 10:55:16.283320 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:16.283229    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22b423d5db5117bad10919b25546cc93b4e7ce2c0e7cf313ecd55afaaa055b38"} err="failed to get container status \"22b423d5db5117bad10919b25546cc93b4e7ce2c0e7cf313ecd55afaaa055b38\": rpc error: code = NotFound desc = could not find container \"22b423d5db5117bad10919b25546cc93b4e7ce2c0e7cf313ecd55afaaa055b38\": container with ID starting with 22b423d5db5117bad10919b25546cc93b4e7ce2c0e7cf313ecd55afaaa055b38 not found: ID does not exist"
Sep 19 10:55:16.283320 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:16.283248    2576 scope.go:117] "RemoveContainer" containerID="5e286408bca4e0bc26339e4c017f8e622d152369a5c67a329a48699d5cc29f29"
Sep 19 10:55:16.283650 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:55:16.283621    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e286408bca4e0bc26339e4c017f8e622d152369a5c67a329a48699d5cc29f29\": container with ID starting with 5e286408bca4e0bc26339e4c017f8e622d152369a5c67a329a48699d5cc29f29 not found: ID does not exist" containerID="5e286408bca4e0bc26339e4c017f8e622d152369a5c67a329a48699d5cc29f29"
Sep 19 10:55:16.283761 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:16.283655    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e286408bca4e0bc26339e4c017f8e622d152369a5c67a329a48699d5cc29f29"} err="failed to get container status \"5e286408bca4e0bc26339e4c017f8e622d152369a5c67a329a48699d5cc29f29\": rpc error: code = NotFound desc = could not find container \"5e286408bca4e0bc26339e4c017f8e622d152369a5c67a329a48699d5cc29f29\": container with ID starting with 5e286408bca4e0bc26339e4c017f8e622d152369a5c67a329a48699d5cc29f29 not found: ID does not exist"
Sep 19 10:55:16.286356 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:16.286335    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"]
Sep 19 10:55:16.289988 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:16.289968    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-e898f8-openshift-default-6bd8759c9b-nzgwg"]
Sep 19 10:55:17.003569 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:55:17.003537    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cadfa4e9-7d31-487d-8b57-4087c39c917a" path="/var/lib/kubelet/pods/cadfa4e9-7d31-487d-8b57-4087c39c917a/volumes"
Sep 19 10:56:15.621471 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.621437    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"]
Sep 19 10:56:15.621923 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.621909    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cadfa4e9-7d31-487d-8b57-4087c39c917a" containerName="istio-proxy"
Sep 19 10:56:15.621969 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.621924    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cadfa4e9-7d31-487d-8b57-4087c39c917a" containerName="istio-proxy"
Sep 19 10:56:15.621969 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.621946    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cadfa4e9-7d31-487d-8b57-4087c39c917a" containerName="istio-proxy"
Sep 19 10:56:15.621969 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.621952    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cadfa4e9-7d31-487d-8b57-4087c39c917a" containerName="istio-proxy"
Sep 19 10:56:15.622086 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.622022    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cadfa4e9-7d31-487d-8b57-4087c39c917a" containerName="istio-proxy"
Sep 19 10:56:15.622086 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.622032    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cadfa4e9-7d31-487d-8b57-4087c39c917a" containerName="istio-proxy"
Sep 19 10:56:15.625385 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.625369    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:15.627905 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.627878    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-5fd2ea\"/\"openshift-service-ca.crt\""
Sep 19 10:56:15.628054 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.627913    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-5fd2ea\"/\"kube-root-ca.crt\""
Sep 19 10:56:15.628054 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.627913    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-5fd2ea\"/\"default-dockercfg-t29pf\""
Sep 19 10:56:15.629158 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.629129    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-5fd2ea\"/\"noten-model-5fd2ea-kserve-self-signed-certs\""
Sep 19 10:56:15.632466 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.632445    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"]
Sep 19 10:56:15.725733 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.725690    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-tmp-dir\") pod \"noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") " pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:15.725902 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.725750    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnnz2\" (UniqueName: \"kubernetes.io/projected/581bc120-8bb5-411a-b9ac-44853b83f35b-kube-api-access-wnnz2\") pod \"noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") " pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:15.725902 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.725793    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-model-cache\") pod \"noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") " pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:15.725902 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.725811    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581bc120-8bb5-411a-b9ac-44853b83f35b-tls-certs\") pod \"noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") " pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:15.725902 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.725826    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-home\") pod \"noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") " pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:15.726074 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.725962    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-dshm\") pod \"noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") " pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:15.826662 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.826633    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnnz2\" (UniqueName: \"kubernetes.io/projected/581bc120-8bb5-411a-b9ac-44853b83f35b-kube-api-access-wnnz2\") pod \"noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") " pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:15.826833 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.826671    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-model-cache\") pod \"noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") " pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:15.826833 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.826692    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581bc120-8bb5-411a-b9ac-44853b83f35b-tls-certs\") pod \"noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") " pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:15.826833 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.826710    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-home\") pod \"noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") " pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:15.826833 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.826789    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-dshm\") pod \"noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") " pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:15.827103 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.826869    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-tmp-dir\") pod \"noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") " pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:15.827103 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.827081    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-model-cache\") pod \"noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") " pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:15.827217 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.827141    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-home\") pod \"noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") " pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:15.827217 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.827212    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-tmp-dir\") pod \"noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") " pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:15.829169 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.829145    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-dshm\") pod \"noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") " pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:15.829502 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.829483    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581bc120-8bb5-411a-b9ac-44853b83f35b-tls-certs\") pod \"noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") " pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:15.834398 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.834376    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnnz2\" (UniqueName: \"kubernetes.io/projected/581bc120-8bb5-411a-b9ac-44853b83f35b-kube-api-access-wnnz2\") pod \"noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") " pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:15.936079 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:15.935985    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:16.208018 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:16.207992    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"]
Sep 19 10:56:16.209727 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:56:16.209699    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod581bc120_8bb5_411a_b9ac_44853b83f35b.slice/crio-db3ad7b89c6438d9eafdc2e15f9ac007c317784823e5898b02923cebee95f303 WatchSource:0}: Error finding container db3ad7b89c6438d9eafdc2e15f9ac007c317784823e5898b02923cebee95f303: Status 404 returned error can't find the container with id db3ad7b89c6438d9eafdc2e15f9ac007c317784823e5898b02923cebee95f303
Sep 19 10:56:16.609073 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:16.609023    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr" event={"ID":"581bc120-8bb5-411a-b9ac-44853b83f35b","Type":"ContainerStarted","Data":"799f11df4db7f8d33618a81f89c86f58dd0c06e2382c618679a86af5c55a2dda"}
Sep 19 10:56:16.609241 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:16.609093    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr" event={"ID":"581bc120-8bb5-411a-b9ac-44853b83f35b","Type":"ContainerStarted","Data":"db3ad7b89c6438d9eafdc2e15f9ac007c317784823e5898b02923cebee95f303"}
Sep 19 10:56:16.625263 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:16.625221    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr" podStartSLOduration=1.625205982 podStartE2EDuration="1.625205982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:56:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:56:16.624485325 +0000 UTC m=+1186.120791419" watchObservedRunningTime="2026-09-19 10:56:16.625205982 +0000 UTC m=+1186.121512077"
Sep 19 10:56:25.936327 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:25.936278    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:25.936327 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:25.936343    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:25.948888 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:25.948854    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:26.675432 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:26.675402    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:56:30.998762 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:30.998736    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 10:56:31.004678 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:31.004655    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 10:56:40.887175 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:40.887140    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"]
Sep 19 10:56:40.887730 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:40.887383    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw" podUID="3c1b3443-b58f-4637-a0fb-53b219d92689" containerName="maas-api" containerID="cri-o://60e777c1ef1e2db3e073a0e166e7aed2b0279499faa086bca876c901cd12e5fa" gracePeriod=30
Sep 19 10:56:41.155464 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.155400    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"
Sep 19 10:56:41.273106 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.273071    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c1b3443-b58f-4637-a0fb-53b219d92689-metrics-certs\") pod \"3c1b3443-b58f-4637-a0fb-53b219d92689\" (UID: \"3c1b3443-b58f-4637-a0fb-53b219d92689\") "
Sep 19 10:56:41.273274 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.273131    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c1b3443-b58f-4637-a0fb-53b219d92689-openshift-service-ca\") pod \"3c1b3443-b58f-4637-a0fb-53b219d92689\" (UID: \"3c1b3443-b58f-4637-a0fb-53b219d92689\") "
Sep 19 10:56:41.273274 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.273180    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwfmx\" (UniqueName: \"kubernetes.io/projected/3c1b3443-b58f-4637-a0fb-53b219d92689-kube-api-access-cwfmx\") pod \"3c1b3443-b58f-4637-a0fb-53b219d92689\" (UID: \"3c1b3443-b58f-4637-a0fb-53b219d92689\") "
Sep 19 10:56:41.273274 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.273219    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c1b3443-b58f-4637-a0fb-53b219d92689-maas-api-tls\") pod \"3c1b3443-b58f-4637-a0fb-53b219d92689\" (UID: \"3c1b3443-b58f-4637-a0fb-53b219d92689\") "
Sep 19 10:56:41.273585 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.273558    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c1b3443-b58f-4637-a0fb-53b219d92689-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3c1b3443-b58f-4637-a0fb-53b219d92689" (UID: "3c1b3443-b58f-4637-a0fb-53b219d92689"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:56:41.275444 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.275410    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c1b3443-b58f-4637-a0fb-53b219d92689-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3c1b3443-b58f-4637-a0fb-53b219d92689" (UID: "3c1b3443-b58f-4637-a0fb-53b219d92689"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:56:41.275573 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.275506    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c1b3443-b58f-4637-a0fb-53b219d92689-kube-api-access-cwfmx" (OuterVolumeSpecName: "kube-api-access-cwfmx") pod "3c1b3443-b58f-4637-a0fb-53b219d92689" (UID: "3c1b3443-b58f-4637-a0fb-53b219d92689"). InnerVolumeSpecName "kube-api-access-cwfmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:56:41.275573 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.275543    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c1b3443-b58f-4637-a0fb-53b219d92689-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3c1b3443-b58f-4637-a0fb-53b219d92689" (UID: "3c1b3443-b58f-4637-a0fb-53b219d92689"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:56:41.374756 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.374725    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwfmx\" (UniqueName: \"kubernetes.io/projected/3c1b3443-b58f-4637-a0fb-53b219d92689-kube-api-access-cwfmx\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:41.374756 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.374750    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c1b3443-b58f-4637-a0fb-53b219d92689-maas-api-tls\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:41.374756 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.374759    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c1b3443-b58f-4637-a0fb-53b219d92689-metrics-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:41.374970 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.374768    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c1b3443-b58f-4637-a0fb-53b219d92689-openshift-service-ca\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:41.747595 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.747562    2576 generic.go:356] "Generic (PLEG): container finished" podID="3c1b3443-b58f-4637-a0fb-53b219d92689" containerID="60e777c1ef1e2db3e073a0e166e7aed2b0279499faa086bca876c901cd12e5fa" exitCode=0
Sep 19 10:56:41.747783 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.747611    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"
Sep 19 10:56:41.747783 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.747634    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw" event={"ID":"3c1b3443-b58f-4637-a0fb-53b219d92689","Type":"ContainerDied","Data":"60e777c1ef1e2db3e073a0e166e7aed2b0279499faa086bca876c901cd12e5fa"}
Sep 19 10:56:41.747783 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.747685    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw" event={"ID":"3c1b3443-b58f-4637-a0fb-53b219d92689","Type":"ContainerDied","Data":"25db93a258988bccbfa39763c587f6f63c6f00d5312eb5e358068a9c7273c1d5"}
Sep 19 10:56:41.747783 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.747708    2576 scope.go:117] "RemoveContainer" containerID="60e777c1ef1e2db3e073a0e166e7aed2b0279499faa086bca876c901cd12e5fa"
Sep 19 10:56:41.758165 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.758091    2576 scope.go:117] "RemoveContainer" containerID="60e777c1ef1e2db3e073a0e166e7aed2b0279499faa086bca876c901cd12e5fa"
Sep 19 10:56:41.758448 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:56:41.758417    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60e777c1ef1e2db3e073a0e166e7aed2b0279499faa086bca876c901cd12e5fa\": container with ID starting with 60e777c1ef1e2db3e073a0e166e7aed2b0279499faa086bca876c901cd12e5fa not found: ID does not exist" containerID="60e777c1ef1e2db3e073a0e166e7aed2b0279499faa086bca876c901cd12e5fa"
Sep 19 10:56:41.758542 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.758447    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60e777c1ef1e2db3e073a0e166e7aed2b0279499faa086bca876c901cd12e5fa"} err="failed to get container status \"60e777c1ef1e2db3e073a0e166e7aed2b0279499faa086bca876c901cd12e5fa\": rpc error: code = NotFound desc = could not find container \"60e777c1ef1e2db3e073a0e166e7aed2b0279499faa086bca876c901cd12e5fa\": container with ID starting with 60e777c1ef1e2db3e073a0e166e7aed2b0279499faa086bca876c901cd12e5fa not found: ID does not exist"
Sep 19 10:56:41.769735 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.769703    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"]
Sep 19 10:56:41.773301 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:41.773281    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0d36-7c56f97d64-g8flw"]
Sep 19 10:56:43.001837 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:43.001807    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c1b3443-b58f-4637-a0fb-53b219d92689" path="/var/lib/kubelet/pods/3c1b3443-b58f-4637-a0fb-53b219d92689/volumes"
Sep 19 10:56:47.741970 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:47.741940    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"]
Sep 19 10:56:47.783154 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:47.783117    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p" podUID="1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" containerName="istio-proxy" containerID="cri-o://911b32135f2bf65ad46996e7745f08d64a9b1d44951577973fb6429aff74ba35" gracePeriod=30
Sep 19 10:56:51.188997 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:51.188964    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"]
Sep 19 10:56:51.189486 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:51.189244    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv" podUID="8b1d66fc-64ad-439f-8618-14ce14194ea8" containerName="istio-proxy" containerID="cri-o://45dbaaada28ad80e662dcecaddd227df952d25cfd3f61c99e250970bc655cfe1" gracePeriod=30
Sep 19 10:56:53.056864 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.056798    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:56:53.191933 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.191910    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p664k\" (UniqueName: \"kubernetes.io/projected/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-kube-api-access-p664k\") pod \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") "
Sep 19 10:56:53.192114 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.191966    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-workload-certs\") pod \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") "
Sep 19 10:56:53.192114 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.191986    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-podinfo\") pod \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") "
Sep 19 10:56:53.192114 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.192010    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-token\") pod \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") "
Sep 19 10:56:53.192320 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.192137    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-credential-socket\") pod \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") "
Sep 19 10:56:53.192320 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.192198    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-workload-socket\") pod \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") "
Sep 19 10:56:53.192320 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.192228    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-envoy\") pod \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") "
Sep 19 10:56:53.192320 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.192291    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" (UID: "1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:53.192526 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.192377    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-data\") pod \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") "
Sep 19 10:56:53.192526 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.192418    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istiod-ca-cert\") pod \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\" (UID: \"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9\") "
Sep 19 10:56:53.192526 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.192477    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" (UID: "1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:53.192676 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.192561    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" (UID: "1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:53.192734 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.192679    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-data" (OuterVolumeSpecName: "istio-data") pod "1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" (UID: "1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:53.192939 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.192902    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" (UID: "1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:56:53.193153 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.193133    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:53.193230 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.193157    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:53.193230 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.193171    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:53.193230 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.193183    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:53.193230 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.193194    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:53.194321 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.194299    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-token" (OuterVolumeSpecName: "istio-token") pod "1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" (UID: "1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:56:53.194402 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.194376    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" (UID: "1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:53.194752 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.194736    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" (UID: "1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 10:56:53.194798 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.194769    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-kube-api-access-p664k" (OuterVolumeSpecName: "kube-api-access-p664k") pod "1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" (UID: "1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9"). InnerVolumeSpecName "kube-api-access-p664k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:56:53.294457 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.294427    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p664k\" (UniqueName: \"kubernetes.io/projected/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-kube-api-access-p664k\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:53.294457 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.294455    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:53.294597 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.294469    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:53.294597 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.294481    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:53.816828 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.816792    2576 generic.go:356] "Generic (PLEG): container finished" podID="1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" containerID="911b32135f2bf65ad46996e7745f08d64a9b1d44951577973fb6429aff74ba35" exitCode=0
Sep 19 10:56:53.817015 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.816844    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"
Sep 19 10:56:53.817015 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.816873    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p" event={"ID":"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9","Type":"ContainerDied","Data":"911b32135f2bf65ad46996e7745f08d64a9b1d44951577973fb6429aff74ba35"}
Sep 19 10:56:53.817015 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.816950    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p" event={"ID":"1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9","Type":"ContainerDied","Data":"7c95ece5330dcebc64d550de79403da35d37c58b70fc9eb6baff334321fbc2e5"}
Sep 19 10:56:53.817015 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.816972    2576 scope.go:117] "RemoveContainer" containerID="911b32135f2bf65ad46996e7745f08d64a9b1d44951577973fb6429aff74ba35"
Sep 19 10:56:53.826978 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.826961    2576 scope.go:117] "RemoveContainer" containerID="34fb6d22a9286df728c08bbd7d5976eec7019b9481b3784a451a6ed7062c9565"
Sep 19 10:56:53.835526 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.835510    2576 scope.go:117] "RemoveContainer" containerID="911b32135f2bf65ad46996e7745f08d64a9b1d44951577973fb6429aff74ba35"
Sep 19 10:56:53.835837 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:56:53.835797    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"911b32135f2bf65ad46996e7745f08d64a9b1d44951577973fb6429aff74ba35\": container with ID starting with 911b32135f2bf65ad46996e7745f08d64a9b1d44951577973fb6429aff74ba35 not found: ID does not exist" containerID="911b32135f2bf65ad46996e7745f08d64a9b1d44951577973fb6429aff74ba35"
Sep 19 10:56:53.835929 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.835844    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"911b32135f2bf65ad46996e7745f08d64a9b1d44951577973fb6429aff74ba35"} err="failed to get container status \"911b32135f2bf65ad46996e7745f08d64a9b1d44951577973fb6429aff74ba35\": rpc error: code = NotFound desc = could not find container \"911b32135f2bf65ad46996e7745f08d64a9b1d44951577973fb6429aff74ba35\": container with ID starting with 911b32135f2bf65ad46996e7745f08d64a9b1d44951577973fb6429aff74ba35 not found: ID does not exist"
Sep 19 10:56:53.835929 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.835865    2576 scope.go:117] "RemoveContainer" containerID="34fb6d22a9286df728c08bbd7d5976eec7019b9481b3784a451a6ed7062c9565"
Sep 19 10:56:53.836247 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:56:53.836218    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34fb6d22a9286df728c08bbd7d5976eec7019b9481b3784a451a6ed7062c9565\": container with ID starting with 34fb6d22a9286df728c08bbd7d5976eec7019b9481b3784a451a6ed7062c9565 not found: ID does not exist" containerID="34fb6d22a9286df728c08bbd7d5976eec7019b9481b3784a451a6ed7062c9565"
Sep 19 10:56:53.836348 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.836258    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34fb6d22a9286df728c08bbd7d5976eec7019b9481b3784a451a6ed7062c9565"} err="failed to get container status \"34fb6d22a9286df728c08bbd7d5976eec7019b9481b3784a451a6ed7062c9565\": rpc error: code = NotFound desc = could not find container \"34fb6d22a9286df728c08bbd7d5976eec7019b9481b3784a451a6ed7062c9565\": container with ID starting with 34fb6d22a9286df728c08bbd7d5976eec7019b9481b3784a451a6ed7062c9565 not found: ID does not exist"
Sep 19 10:56:53.838786 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.838768    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"]
Sep 19 10:56:53.842565 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:53.842547    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-6a0d36-openshift-default-6d675d7d6b-c2w9p"]
Sep 19 10:56:55.001808 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.001776    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" path="/var/lib/kubelet/pods/1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9/volumes"
Sep 19 10:56:55.313201 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.313128    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"]
Sep 19 10:56:55.313855 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.313834    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c1b3443-b58f-4637-a0fb-53b219d92689" containerName="maas-api"
Sep 19 10:56:55.313898 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.313862    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c1b3443-b58f-4637-a0fb-53b219d92689" containerName="maas-api"
Sep 19 10:56:55.313898 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.313882    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" containerName="istio-proxy"
Sep 19 10:56:55.313898 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.313891    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" containerName="istio-proxy"
Sep 19 10:56:55.313991 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.313933    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" containerName="istio-proxy"
Sep 19 10:56:55.313991 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.313942    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" containerName="istio-proxy"
Sep 19 10:56:55.314079 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.314070    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" containerName="istio-proxy"
Sep 19 10:56:55.314119 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.314095    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c1b3443-b58f-4637-a0fb-53b219d92689" containerName="maas-api"
Sep 19 10:56:55.319256 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.319231    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.321962 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.321938    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"]
Sep 19 10:56:55.322159 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.322140    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-5116ec\"/\"openshift-service-ca.crt\""
Sep 19 10:56:55.323318 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.323291    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-5116ec\"/\"default-dockercfg-ghbth\""
Sep 19 10:56:55.323428 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.323320    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-5116ec\"/\"e2e-conflict-b-9f7696ca-kserve-self-signed-certs\""
Sep 19 10:56:55.323627 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.323613    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-5116ec\"/\"kube-root-ca.crt\""
Sep 19 10:56:55.410058 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.410005    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-model-cache\") pod \"e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") " pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.410239 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.410065    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-home\") pod \"e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") " pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.410239 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.410097    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-tmp-dir\") pod \"e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") " pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.410239 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.410209    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8ac3298-0bff-4317-a562-34c73da187e8-tls-certs\") pod \"e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") " pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.410424 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.410253    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-dshm\") pod \"e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") " pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.410424 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.410284    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7hjp\" (UniqueName: \"kubernetes.io/projected/b8ac3298-0bff-4317-a562-34c73da187e8-kube-api-access-w7hjp\") pod \"e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") " pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.510901 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.510865    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-home\") pod \"e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") " pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.510901 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.510902    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-tmp-dir\") pod \"e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") " pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.511169 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.510948    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8ac3298-0bff-4317-a562-34c73da187e8-tls-certs\") pod \"e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") " pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.511169 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.510968    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-dshm\") pod \"e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") " pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.511169 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.510988    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7hjp\" (UniqueName: \"kubernetes.io/projected/b8ac3298-0bff-4317-a562-34c73da187e8-kube-api-access-w7hjp\") pod \"e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") " pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.511169 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.511089    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-model-cache\") pod \"e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") " pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.511380 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.511249    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-home\") pod \"e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") " pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.511428 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.511382    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-tmp-dir\") pod \"e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") " pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.511466 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.511449    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-model-cache\") pod \"e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") " pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.513383 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.513364    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-dshm\") pod \"e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") " pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.513631 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.513614    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8ac3298-0bff-4317-a562-34c73da187e8-tls-certs\") pod \"e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") " pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.518168 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.518147    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7hjp\" (UniqueName: \"kubernetes.io/projected/b8ac3298-0bff-4317-a562-34c73da187e8-kube-api-access-w7hjp\") pod \"e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") " pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.630833 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.630803    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:56:55.909863 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:55.909838    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"]
Sep 19 10:56:55.911540 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:56:55.911510    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8ac3298_0bff_4317_a562_34c73da187e8.slice/crio-8d0d2ed5bf74946b0e5e81e8bfa203de8b68ef65580f9b6a3cec52a594f106e1 WatchSource:0}: Error finding container 8d0d2ed5bf74946b0e5e81e8bfa203de8b68ef65580f9b6a3cec52a594f106e1: Status 404 returned error can't find the container with id 8d0d2ed5bf74946b0e5e81e8bfa203de8b68ef65580f9b6a3cec52a594f106e1
Sep 19 10:56:56.432212 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.432187    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:56:56.513135 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.513069    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"]
Sep 19 10:56:56.529141 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.529112    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-workload-certs\") pod \"8b1d66fc-64ad-439f-8618-14ce14194ea8\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") "
Sep 19 10:56:56.529264 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.529168    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-credential-socket\") pod \"8b1d66fc-64ad-439f-8618-14ce14194ea8\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") "
Sep 19 10:56:56.529264 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.529229    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-workload-socket\") pod \"8b1d66fc-64ad-439f-8618-14ce14194ea8\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") "
Sep 19 10:56:56.529387 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.529284    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-envoy\") pod \"8b1d66fc-64ad-439f-8618-14ce14194ea8\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") "
Sep 19 10:56:56.529387 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.529338    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsz27\" (UniqueName: \"kubernetes.io/projected/8b1d66fc-64ad-439f-8618-14ce14194ea8-kube-api-access-rsz27\") pod \"8b1d66fc-64ad-439f-8618-14ce14194ea8\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") "
Sep 19 10:56:56.529505 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.529394    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-data\") pod \"8b1d66fc-64ad-439f-8618-14ce14194ea8\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") "
Sep 19 10:56:56.529505 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.529436    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-token\") pod \"8b1d66fc-64ad-439f-8618-14ce14194ea8\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") "
Sep 19 10:56:56.529505 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.529495    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b1d66fc-64ad-439f-8618-14ce14194ea8-istiod-ca-cert\") pod \"8b1d66fc-64ad-439f-8618-14ce14194ea8\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") "
Sep 19 10:56:56.529666 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.529547    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-podinfo\") pod \"8b1d66fc-64ad-439f-8618-14ce14194ea8\" (UID: \"8b1d66fc-64ad-439f-8618-14ce14194ea8\") "
Sep 19 10:56:56.531394 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.531176    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b1d66fc-64ad-439f-8618-14ce14194ea8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8b1d66fc-64ad-439f-8618-14ce14194ea8" (UID: "8b1d66fc-64ad-439f-8618-14ce14194ea8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:56:56.531394 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.531269    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8b1d66fc-64ad-439f-8618-14ce14194ea8" (UID: "8b1d66fc-64ad-439f-8618-14ce14194ea8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:56.531394 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.531292    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-data" (OuterVolumeSpecName: "istio-data") pod "8b1d66fc-64ad-439f-8618-14ce14194ea8" (UID: "8b1d66fc-64ad-439f-8618-14ce14194ea8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:56.531763 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.531736    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8b1d66fc-64ad-439f-8618-14ce14194ea8" (UID: "8b1d66fc-64ad-439f-8618-14ce14194ea8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:56.532096 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.532073    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8b1d66fc-64ad-439f-8618-14ce14194ea8" (UID: "8b1d66fc-64ad-439f-8618-14ce14194ea8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:56.532767 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.532746    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8b1d66fc-64ad-439f-8618-14ce14194ea8" (UID: "8b1d66fc-64ad-439f-8618-14ce14194ea8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:56.532876 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.532857    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8b1d66fc-64ad-439f-8618-14ce14194ea8" (UID: "8b1d66fc-64ad-439f-8618-14ce14194ea8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 10:56:56.533720 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.533698    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-token" (OuterVolumeSpecName: "istio-token") pod "8b1d66fc-64ad-439f-8618-14ce14194ea8" (UID: "8b1d66fc-64ad-439f-8618-14ce14194ea8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:56:56.534222 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.534195    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b1d66fc-64ad-439f-8618-14ce14194ea8-kube-api-access-rsz27" (OuterVolumeSpecName: "kube-api-access-rsz27") pod "8b1d66fc-64ad-439f-8618-14ce14194ea8" (UID: "8b1d66fc-64ad-439f-8618-14ce14194ea8"). InnerVolumeSpecName "kube-api-access-rsz27". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:56:56.631141 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.631119    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:56.631141 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.631142    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsz27\" (UniqueName: \"kubernetes.io/projected/8b1d66fc-64ad-439f-8618-14ce14194ea8-kube-api-access-rsz27\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:56.631275 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.631151    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:56.631275 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.631159    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:56.631275 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.631166    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b1d66fc-64ad-439f-8618-14ce14194ea8-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:56.631275 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.631173    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b1d66fc-64ad-439f-8618-14ce14194ea8-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:56.631275 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.631180    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:56.631275 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.631188    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:56.631275 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.631196    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1d66fc-64ad-439f-8618-14ce14194ea8-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:56:56.836272 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.836236    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q" event={"ID":"b8ac3298-0bff-4317-a562-34c73da187e8","Type":"ContainerStarted","Data":"718de77f5a48d39e553029e83b37df3b6ffed2b80c3638a31cdf12e02a130a36"}
Sep 19 10:56:56.836415 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.836282    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q" event={"ID":"b8ac3298-0bff-4317-a562-34c73da187e8","Type":"ContainerStarted","Data":"8d0d2ed5bf74946b0e5e81e8bfa203de8b68ef65580f9b6a3cec52a594f106e1"}
Sep 19 10:56:56.837564 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.837538    2576 generic.go:356] "Generic (PLEG): container finished" podID="8b1d66fc-64ad-439f-8618-14ce14194ea8" containerID="45dbaaada28ad80e662dcecaddd227df952d25cfd3f61c99e250970bc655cfe1" exitCode=0
Sep 19 10:56:56.837677 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.837592    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"
Sep 19 10:56:56.837677 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.837613    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv" event={"ID":"8b1d66fc-64ad-439f-8618-14ce14194ea8","Type":"ContainerDied","Data":"45dbaaada28ad80e662dcecaddd227df952d25cfd3f61c99e250970bc655cfe1"}
Sep 19 10:56:56.837751 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.837678    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv" event={"ID":"8b1d66fc-64ad-439f-8618-14ce14194ea8","Type":"ContainerDied","Data":"e4e0933ce36314e2baccc99f70e02f68032cfee3ae9013a1a3d2ba1922e832ef"}
Sep 19 10:56:56.837751 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.837701    2576 scope.go:117] "RemoveContainer" containerID="45dbaaada28ad80e662dcecaddd227df952d25cfd3f61c99e250970bc655cfe1"
Sep 19 10:56:56.847153 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.847134    2576 scope.go:117] "RemoveContainer" containerID="45dbaaada28ad80e662dcecaddd227df952d25cfd3f61c99e250970bc655cfe1"
Sep 19 10:56:56.847430 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:56:56.847414    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45dbaaada28ad80e662dcecaddd227df952d25cfd3f61c99e250970bc655cfe1\": container with ID starting with 45dbaaada28ad80e662dcecaddd227df952d25cfd3f61c99e250970bc655cfe1 not found: ID does not exist" containerID="45dbaaada28ad80e662dcecaddd227df952d25cfd3f61c99e250970bc655cfe1"
Sep 19 10:56:56.847483 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.847437    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45dbaaada28ad80e662dcecaddd227df952d25cfd3f61c99e250970bc655cfe1"} err="failed to get container status \"45dbaaada28ad80e662dcecaddd227df952d25cfd3f61c99e250970bc655cfe1\": rpc error: code = NotFound desc = could not find container \"45dbaaada28ad80e662dcecaddd227df952d25cfd3f61c99e250970bc655cfe1\": container with ID starting with 45dbaaada28ad80e662dcecaddd227df952d25cfd3f61c99e250970bc655cfe1 not found: ID does not exist"
Sep 19 10:56:56.852221 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.852182    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q" podStartSLOduration=1.852168347 podStartE2EDuration="1.852168347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:56:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:56:56.850809641 +0000 UTC m=+1226.347115733" watchObservedRunningTime="2026-09-19 10:56:56.852168347 +0000 UTC m=+1226.348474438"
Sep 19 10:56:56.864657 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.864629    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"]
Sep 19 10:56:56.870212 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:56.870189    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-5fd2ea-openshift-default-5d9f658854-5mlgv"]
Sep 19 10:56:57.001604 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:56:57.001574    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b1d66fc-64ad-439f-8618-14ce14194ea8" path="/var/lib/kubelet/pods/8b1d66fc-64ad-439f-8618-14ce14194ea8/volumes"
Sep 19 10:57:05.631484 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.631448    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:57:05.631923 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.631504    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:57:05.644460 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.644417    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:57:05.682551 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.682525    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"]
Sep 19 10:57:05.683036 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.683022    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b1d66fc-64ad-439f-8618-14ce14194ea8" containerName="istio-proxy"
Sep 19 10:57:05.683094 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.683056    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b1d66fc-64ad-439f-8618-14ce14194ea8" containerName="istio-proxy"
Sep 19 10:57:05.683163 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.683154    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b1d66fc-64ad-439f-8618-14ce14194ea8" containerName="istio-proxy"
Sep 19 10:57:05.683196 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.683165    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1aa3bf86-e87d-4ba9-b4ce-789c766e2cc9" containerName="istio-proxy"
Sep 19 10:57:05.686374 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.686360    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.688686 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.688667    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-137f88-openshift-default-dockercfg-6jfkv\""
Sep 19 10:57:05.696646 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.696620    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"]
Sep 19 10:57:05.819775 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.819748    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-credential-socket\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.819933 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.819781    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-podinfo\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.819933 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.819818    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-data\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.819933 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.819882    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3142ef93-54d6-463d-985f-6fb7b28eadfe-istiod-ca-cert\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.819933 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.819923    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-token\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.820124 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.819959    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rntr8\" (UniqueName: \"kubernetes.io/projected/3142ef93-54d6-463d-985f-6fb7b28eadfe-kube-api-access-rntr8\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.820124 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.820028    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-envoy\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.820124 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.820082    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-workload-certs\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.820124 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.820112    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-workload-socket\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.903370 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.903295    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:57:05.920532 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.920505    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-workload-socket\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.920655 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.920556    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-credential-socket\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.920655 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.920577    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-podinfo\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.920655 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.920617    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-data\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.920655 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.920648    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3142ef93-54d6-463d-985f-6fb7b28eadfe-istiod-ca-cert\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.920877 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.920680    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-token\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.920877 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.920718    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rntr8\" (UniqueName: \"kubernetes.io/projected/3142ef93-54d6-463d-985f-6fb7b28eadfe-kube-api-access-rntr8\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.920877 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.920779    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-envoy\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.920877 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.920828    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-workload-certs\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.921117 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.920898    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-workload-socket\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.921117 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.921084    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-data\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.921226 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.921128    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-credential-socket\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.921467 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.921443    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-workload-certs\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.921706 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.921678    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3142ef93-54d6-463d-985f-6fb7b28eadfe-istiod-ca-cert\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.923266 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.923247    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-podinfo\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.923373 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.923354    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-envoy\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.932162 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.932114    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-token\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.932571 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.932551    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rntr8\" (UniqueName: \"kubernetes.io/projected/3142ef93-54d6-463d-985f-6fb7b28eadfe-kube-api-access-rntr8\") pod \"e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") " pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:05.998806 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:05.998778    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:06.291899 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:06.291865    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"]
Sep 19 10:57:06.299029 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:06.298994    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:57:06.299144 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:06.299102    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:57:06.299144 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:06.299137    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:57:06.898310 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:06.898269    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l" event={"ID":"3142ef93-54d6-463d-985f-6fb7b28eadfe","Type":"ContainerStarted","Data":"67bacaa6732578b422d3b1e7ce6a4c8d59cc42cd93c26be2e176951431fb8b27"}
Sep 19 10:57:06.898724 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:06.898339    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l" event={"ID":"3142ef93-54d6-463d-985f-6fb7b28eadfe","Type":"ContainerStarted","Data":"5689f91bd590219498fd5b6e99bbf730e858a7eede8c5d02298e786495a19543"}
Sep 19 10:57:06.920107 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:06.920058    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l" podStartSLOduration=1.9200243320000001 podStartE2EDuration="1.920024332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:57:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:57:06.91826276 +0000 UTC m=+1236.414568876" watchObservedRunningTime="2026-09-19 10:57:06.920024332 +0000 UTC m=+1236.416330423"
Sep 19 10:57:07.002813 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:07.002781    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:08.004884 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:08.004855    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:08.909240 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:08.909216    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:08.910143 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:08.910125    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:11.538910 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.538870    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr" podUID="581bc120-8bb5-411a-b9ac-44853b83f35b" containerName="main" containerID="cri-o://799f11df4db7f8d33618a81f89c86f58dd0c06e2382c618679a86af5c55a2dda" gracePeriod=45
Sep 19 10:57:11.802138 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.802085    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:57:11.885790 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.885763    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581bc120-8bb5-411a-b9ac-44853b83f35b-tls-certs\") pod \"581bc120-8bb5-411a-b9ac-44853b83f35b\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") "
Sep 19 10:57:11.885946 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.885816    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-model-cache\") pod \"581bc120-8bb5-411a-b9ac-44853b83f35b\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") "
Sep 19 10:57:11.885946 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.885834    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-home\") pod \"581bc120-8bb5-411a-b9ac-44853b83f35b\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") "
Sep 19 10:57:11.885946 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.885861    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-tmp-dir\") pod \"581bc120-8bb5-411a-b9ac-44853b83f35b\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") "
Sep 19 10:57:11.885946 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.885876    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-dshm\") pod \"581bc120-8bb5-411a-b9ac-44853b83f35b\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") "
Sep 19 10:57:11.885946 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.885907    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnnz2\" (UniqueName: \"kubernetes.io/projected/581bc120-8bb5-411a-b9ac-44853b83f35b-kube-api-access-wnnz2\") pod \"581bc120-8bb5-411a-b9ac-44853b83f35b\" (UID: \"581bc120-8bb5-411a-b9ac-44853b83f35b\") "
Sep 19 10:57:11.886256 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.886093    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-model-cache" (OuterVolumeSpecName: "model-cache") pod "581bc120-8bb5-411a-b9ac-44853b83f35b" (UID: "581bc120-8bb5-411a-b9ac-44853b83f35b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:57:11.886256 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.886121    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-home" (OuterVolumeSpecName: "home") pod "581bc120-8bb5-411a-b9ac-44853b83f35b" (UID: "581bc120-8bb5-411a-b9ac-44853b83f35b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:57:11.886339 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.886294    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "581bc120-8bb5-411a-b9ac-44853b83f35b" (UID: "581bc120-8bb5-411a-b9ac-44853b83f35b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:57:11.886478 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.886456    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-model-cache\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:57:11.886478 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.886479    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-home\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:57:11.886810 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.886492    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-tmp-dir\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:57:11.888389 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.888364    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/581bc120-8bb5-411a-b9ac-44853b83f35b-kube-api-access-wnnz2" (OuterVolumeSpecName: "kube-api-access-wnnz2") pod "581bc120-8bb5-411a-b9ac-44853b83f35b" (UID: "581bc120-8bb5-411a-b9ac-44853b83f35b"). InnerVolumeSpecName "kube-api-access-wnnz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:57:11.888482 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.888434    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-dshm" (OuterVolumeSpecName: "dshm") pod "581bc120-8bb5-411a-b9ac-44853b83f35b" (UID: "581bc120-8bb5-411a-b9ac-44853b83f35b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:57:11.888482 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.888453    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/581bc120-8bb5-411a-b9ac-44853b83f35b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "581bc120-8bb5-411a-b9ac-44853b83f35b" (UID: "581bc120-8bb5-411a-b9ac-44853b83f35b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:57:11.927733 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.927695    2576 generic.go:356] "Generic (PLEG): container finished" podID="581bc120-8bb5-411a-b9ac-44853b83f35b" containerID="799f11df4db7f8d33618a81f89c86f58dd0c06e2382c618679a86af5c55a2dda" exitCode=0
Sep 19 10:57:11.927828 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.927745    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"
Sep 19 10:57:11.927828 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.927768    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr" event={"ID":"581bc120-8bb5-411a-b9ac-44853b83f35b","Type":"ContainerDied","Data":"799f11df4db7f8d33618a81f89c86f58dd0c06e2382c618679a86af5c55a2dda"}
Sep 19 10:57:11.927828 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.927808    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr" event={"ID":"581bc120-8bb5-411a-b9ac-44853b83f35b","Type":"ContainerDied","Data":"db3ad7b89c6438d9eafdc2e15f9ac007c317784823e5898b02923cebee95f303"}
Sep 19 10:57:11.927971 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.927830    2576 scope.go:117] "RemoveContainer" containerID="799f11df4db7f8d33618a81f89c86f58dd0c06e2382c618679a86af5c55a2dda"
Sep 19 10:57:11.940467 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.940450    2576 scope.go:117] "RemoveContainer" containerID="799f11df4db7f8d33618a81f89c86f58dd0c06e2382c618679a86af5c55a2dda"
Sep 19 10:57:11.940799 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:57:11.940774    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"799f11df4db7f8d33618a81f89c86f58dd0c06e2382c618679a86af5c55a2dda\": container with ID starting with 799f11df4db7f8d33618a81f89c86f58dd0c06e2382c618679a86af5c55a2dda not found: ID does not exist" containerID="799f11df4db7f8d33618a81f89c86f58dd0c06e2382c618679a86af5c55a2dda"
Sep 19 10:57:11.940889 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.940810    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"799f11df4db7f8d33618a81f89c86f58dd0c06e2382c618679a86af5c55a2dda"} err="failed to get container status \"799f11df4db7f8d33618a81f89c86f58dd0c06e2382c618679a86af5c55a2dda\": rpc error: code = NotFound desc = could not find container \"799f11df4db7f8d33618a81f89c86f58dd0c06e2382c618679a86af5c55a2dda\": container with ID starting with 799f11df4db7f8d33618a81f89c86f58dd0c06e2382c618679a86af5c55a2dda not found: ID does not exist"
Sep 19 10:57:11.951991 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.951971    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"]
Sep 19 10:57:11.954145 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.954124    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-5fd2ea/noten-model-5fd2ea-kserve-bb6fcdb7f-bl2fr"]
Sep 19 10:57:11.987412 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.987392    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/581bc120-8bb5-411a-b9ac-44853b83f35b-dshm\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:57:11.987412 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.987411    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnnz2\" (UniqueName: \"kubernetes.io/projected/581bc120-8bb5-411a-b9ac-44853b83f35b-kube-api-access-wnnz2\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:57:11.987526 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:11.987420    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581bc120-8bb5-411a-b9ac-44853b83f35b-tls-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:57:12.292453 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.292426    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"]
Sep 19 10:57:12.292909 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.292896    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="581bc120-8bb5-411a-b9ac-44853b83f35b" containerName="main"
Sep 19 10:57:12.292957 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.292912    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="581bc120-8bb5-411a-b9ac-44853b83f35b" containerName="main"
Sep 19 10:57:12.293066 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.292997    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="581bc120-8bb5-411a-b9ac-44853b83f35b" containerName="main"
Sep 19 10:57:12.297373 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.297354    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"
Sep 19 10:57:12.299519 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.299448    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-137f88-serving-cert\""
Sep 19 10:57:12.307600 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.307508    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"]
Sep 19 10:57:12.392120 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.392094    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22gg9\" (UniqueName: \"kubernetes.io/projected/26a85e00-629b-4d45-8984-377f1e13123b-kube-api-access-22gg9\") pod \"maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4\" (UID: \"26a85e00-629b-4d45-8984-377f1e13123b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"
Sep 19 10:57:12.392287 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.392239    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26a85e00-629b-4d45-8984-377f1e13123b-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4\" (UID: \"26a85e00-629b-4d45-8984-377f1e13123b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"
Sep 19 10:57:12.392343 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.392311    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26a85e00-629b-4d45-8984-377f1e13123b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4\" (UID: \"26a85e00-629b-4d45-8984-377f1e13123b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"
Sep 19 10:57:12.392389 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.392345    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26a85e00-629b-4d45-8984-377f1e13123b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4\" (UID: \"26a85e00-629b-4d45-8984-377f1e13123b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"
Sep 19 10:57:12.493309 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.493274    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26a85e00-629b-4d45-8984-377f1e13123b-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4\" (UID: \"26a85e00-629b-4d45-8984-377f1e13123b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"
Sep 19 10:57:12.493483 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.493312    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26a85e00-629b-4d45-8984-377f1e13123b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4\" (UID: \"26a85e00-629b-4d45-8984-377f1e13123b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"
Sep 19 10:57:12.493483 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.493380    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26a85e00-629b-4d45-8984-377f1e13123b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4\" (UID: \"26a85e00-629b-4d45-8984-377f1e13123b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"
Sep 19 10:57:12.493483 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.493463    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22gg9\" (UniqueName: \"kubernetes.io/projected/26a85e00-629b-4d45-8984-377f1e13123b-kube-api-access-22gg9\") pod \"maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4\" (UID: \"26a85e00-629b-4d45-8984-377f1e13123b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"
Sep 19 10:57:12.494134 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.494112    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26a85e00-629b-4d45-8984-377f1e13123b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4\" (UID: \"26a85e00-629b-4d45-8984-377f1e13123b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"
Sep 19 10:57:12.495801 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.495784    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26a85e00-629b-4d45-8984-377f1e13123b-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4\" (UID: \"26a85e00-629b-4d45-8984-377f1e13123b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"
Sep 19 10:57:12.495867 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.495819    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26a85e00-629b-4d45-8984-377f1e13123b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4\" (UID: \"26a85e00-629b-4d45-8984-377f1e13123b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"
Sep 19 10:57:12.505820 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.505797    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22gg9\" (UniqueName: \"kubernetes.io/projected/26a85e00-629b-4d45-8984-377f1e13123b-kube-api-access-22gg9\") pod \"maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4\" (UID: \"26a85e00-629b-4d45-8984-377f1e13123b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"
Sep 19 10:57:12.610646 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.610628    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"
Sep 19 10:57:12.883492 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.883465    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"]
Sep 19 10:57:12.885569 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:57:12.885533    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26a85e00_629b_4d45_8984_377f1e13123b.slice/crio-34932a526d32f627e99cb9e6a0fe9068cac2117834f3b568a67abb65f1753989 WatchSource:0}: Error finding container 34932a526d32f627e99cb9e6a0fe9068cac2117834f3b568a67abb65f1753989: Status 404 returned error can't find the container with id 34932a526d32f627e99cb9e6a0fe9068cac2117834f3b568a67abb65f1753989
Sep 19 10:57:12.935289 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:12.935259    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4" event={"ID":"26a85e00-629b-4d45-8984-377f1e13123b","Type":"ContainerStarted","Data":"34932a526d32f627e99cb9e6a0fe9068cac2117834f3b568a67abb65f1753989"}
Sep 19 10:57:13.003569 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:13.003538    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="581bc120-8bb5-411a-b9ac-44853b83f35b" path="/var/lib/kubelet/pods/581bc120-8bb5-411a-b9ac-44853b83f35b/volumes"
Sep 19 10:57:13.943864 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:13.943826    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4" event={"ID":"26a85e00-629b-4d45-8984-377f1e13123b","Type":"ContainerStarted","Data":"3f0cc357162ff0f15e0828e445bffa96e8ad4302bb7cb7bdb9b67cd78741ab81"}
Sep 19 10:57:13.944331 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:13.943940    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"
Sep 19 10:57:13.960646 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:13.960600    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4" podStartSLOduration=1.4602743280000001 podStartE2EDuration="1.960586412s" totalImagesPullingTime="500.312084ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:57:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:57:13.958331932 +0000 UTC m=+1243.454638025" watchObservedRunningTime="2026-09-19 10:57:13.960586412 +0000 UTC m=+1243.456892520"
Sep 19 10:57:19.955199 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:19.955168    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"
Sep 19 10:57:33.052877 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:33.052845    2576 generic.go:356] "Generic (PLEG): container finished" podID="3142ef93-54d6-463d-985f-6fb7b28eadfe" containerID="67bacaa6732578b422d3b1e7ce6a4c8d59cc42cd93c26be2e176951431fb8b27" exitCode=0
Sep 19 10:57:33.053304 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:33.052917    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l" event={"ID":"3142ef93-54d6-463d-985f-6fb7b28eadfe","Type":"ContainerDied","Data":"67bacaa6732578b422d3b1e7ce6a4c8d59cc42cd93c26be2e176951431fb8b27"}
Sep 19 10:57:33.053354 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:33.053314    2576 scope.go:117] "RemoveContainer" containerID="67bacaa6732578b422d3b1e7ce6a4c8d59cc42cd93c26be2e176951431fb8b27"
Sep 19 10:57:33.054478 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:33.054442    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:57:33.054591 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:33.054526    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:57:33.054591 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:33.054572    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:57:34.064624 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:34.064583    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l" event={"ID":"3142ef93-54d6-463d-985f-6fb7b28eadfe","Type":"ContainerStarted","Data":"f4e78a4853f820c602f6772be9530807b335834a955d18c072056e1b3ecc0013"}
Sep 19 10:57:35.000589 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:35.000556    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 19 10:57:35.000770 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:35.000602    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l" podUID="3142ef93-54d6-463d-985f-6fb7b28eadfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 19 10:57:35.001923 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:35.001896    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:35.999527 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:35.999495    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 19 10:57:35.999983 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:35.999547    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l" podUID="3142ef93-54d6-463d-985f-6fb7b28eadfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 19 10:57:36.999301 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:36.999270    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 19 10:57:36.999499 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:36.999338    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l" podUID="3142ef93-54d6-463d-985f-6fb7b28eadfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 19 10:57:37.999333 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:37.999300    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 19 10:57:37.999734 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:37.999349    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l" podUID="3142ef93-54d6-463d-985f-6fb7b28eadfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 19 10:57:38.909212 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:38.909177    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:38.999606 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:38.999570    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 19 10:57:38.999990 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:38.999641    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l" podUID="3142ef93-54d6-463d-985f-6fb7b28eadfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 19 10:57:39.657595 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.657562    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"]
Sep 19 10:57:39.661419 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.661400    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.663878 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.663860    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-9b3bef-openshift-default-dockercfg-dl4h7\""
Sep 19 10:57:39.673729 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.673708    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"]
Sep 19 10:57:39.789851 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.789813    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpz6b\" (UniqueName: \"kubernetes.io/projected/5915405f-7676-421d-9230-b3c30b27a115-kube-api-access-qpz6b\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.790076 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.789876    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-istio-data\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.790076 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.789915    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-workload-socket\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.790076 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.789987    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5915405f-7676-421d-9230-b3c30b27a115-istio-token\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.790076 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.790016    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5915405f-7676-421d-9230-b3c30b27a115-istiod-ca-cert\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.790076 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.790071    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-credential-socket\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.790348 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.790141    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5915405f-7676-421d-9230-b3c30b27a115-istio-podinfo\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.790348 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.790184    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-istio-envoy\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.790348 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.790221    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-workload-certs\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.890684 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.890656    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5915405f-7676-421d-9230-b3c30b27a115-istio-token\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.890841 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.890690    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5915405f-7676-421d-9230-b3c30b27a115-istiod-ca-cert\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.890841 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.890714    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-credential-socket\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.890841 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.890768    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5915405f-7676-421d-9230-b3c30b27a115-istio-podinfo\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.890841 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.890808    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-istio-envoy\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.891090 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.890845    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-workload-certs\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.891090 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.890908    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpz6b\" (UniqueName: \"kubernetes.io/projected/5915405f-7676-421d-9230-b3c30b27a115-kube-api-access-qpz6b\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.891090 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.890961    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-istio-data\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.891090 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.890991    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-workload-socket\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.891298 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.891220    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-credential-socket\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.891589 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.891395    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-workload-socket\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.891589 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.891479    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-istio-data\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.891589 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.891515    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5915405f-7676-421d-9230-b3c30b27a115-istiod-ca-cert\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.891779 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.891631    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-workload-certs\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.893729 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.893704    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-istio-envoy\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.893729 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.893718    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5915405f-7676-421d-9230-b3c30b27a115-istio-podinfo\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.900446 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.900418    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpz6b\" (UniqueName: \"kubernetes.io/projected/5915405f-7676-421d-9230-b3c30b27a115-kube-api-access-qpz6b\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.900980 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.900955    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5915405f-7676-421d-9230-b3c30b27a115-istio-token\") pod \"e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") " pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:39.973646 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:39.973569    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:40.004455 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:40.004431    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:40.005583 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:40.005561    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 10:57:40.250856 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:40.250829    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"]
Sep 19 10:57:40.252238 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:57:40.252213    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5915405f_7676_421d_9230_b3c30b27a115.slice/crio-59fed62dfa2fb2256087e27d5b5a5253007f73ad6487b993f8fa6e41661e4c9c WatchSource:0}: Error finding container 59fed62dfa2fb2256087e27d5b5a5253007f73ad6487b993f8fa6e41661e4c9c: Status 404 returned error can't find the container with id 59fed62dfa2fb2256087e27d5b5a5253007f73ad6487b993f8fa6e41661e4c9c
Sep 19 10:57:40.254657 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:40.254624    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:57:40.254737 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:40.254690    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:57:40.254737 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:40.254724    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:57:41.107242 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:41.107206    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd" event={"ID":"5915405f-7676-421d-9230-b3c30b27a115","Type":"ContainerStarted","Data":"201b6f91475dbaf0514b38f1f4bf6f5b53552f9ea01cbf117c00c5742c63c6a4"}
Sep 19 10:57:41.107628 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:41.107262    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd" event={"ID":"5915405f-7676-421d-9230-b3c30b27a115","Type":"ContainerStarted","Data":"59fed62dfa2fb2256087e27d5b5a5253007f73ad6487b993f8fa6e41661e4c9c"}
Sep 19 10:57:41.129630 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:41.129576    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd" podStartSLOduration=2.129554795 podStartE2EDuration="2.129554795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:57:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:57:41.125097426 +0000 UTC m=+1270.621403519" watchObservedRunningTime="2026-09-19 10:57:41.129554795 +0000 UTC m=+1270.625860888"
Sep 19 10:57:41.974123 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:41.974088    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:41.980188 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:41.980165    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:42.113979 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:42.113948    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:42.115116 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:42.115094    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:57:47.515466 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:47.515380    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"]
Sep 19 10:57:47.519624 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:47.519600    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"
Sep 19 10:57:47.522021 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:47.522000    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-9b3bef-serving-cert\""
Sep 19 10:57:47.529064 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:47.529015    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"]
Sep 19 10:57:47.674372 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:47.674332    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0933555e-5813-45a1-bbbd-0bd17362423b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz\" (UID: \"0933555e-5813-45a1-bbbd-0bd17362423b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"
Sep 19 10:57:47.674551 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:47.674387    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdww6\" (UniqueName: \"kubernetes.io/projected/0933555e-5813-45a1-bbbd-0bd17362423b-kube-api-access-gdww6\") pod \"maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz\" (UID: \"0933555e-5813-45a1-bbbd-0bd17362423b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"
Sep 19 10:57:47.674551 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:47.674473    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0933555e-5813-45a1-bbbd-0bd17362423b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz\" (UID: \"0933555e-5813-45a1-bbbd-0bd17362423b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"
Sep 19 10:57:47.674551 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:47.674539    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0933555e-5813-45a1-bbbd-0bd17362423b-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz\" (UID: \"0933555e-5813-45a1-bbbd-0bd17362423b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"
Sep 19 10:57:47.775156 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:47.775079    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0933555e-5813-45a1-bbbd-0bd17362423b-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz\" (UID: \"0933555e-5813-45a1-bbbd-0bd17362423b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"
Sep 19 10:57:47.775156 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:47.775128    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0933555e-5813-45a1-bbbd-0bd17362423b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz\" (UID: \"0933555e-5813-45a1-bbbd-0bd17362423b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"
Sep 19 10:57:47.775348 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:47.775169    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdww6\" (UniqueName: \"kubernetes.io/projected/0933555e-5813-45a1-bbbd-0bd17362423b-kube-api-access-gdww6\") pod \"maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz\" (UID: \"0933555e-5813-45a1-bbbd-0bd17362423b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"
Sep 19 10:57:47.775348 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:47.775241    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0933555e-5813-45a1-bbbd-0bd17362423b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz\" (UID: \"0933555e-5813-45a1-bbbd-0bd17362423b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"
Sep 19 10:57:47.775894 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:47.775869    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0933555e-5813-45a1-bbbd-0bd17362423b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz\" (UID: \"0933555e-5813-45a1-bbbd-0bd17362423b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"
Sep 19 10:57:47.777633 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:47.777604    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0933555e-5813-45a1-bbbd-0bd17362423b-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz\" (UID: \"0933555e-5813-45a1-bbbd-0bd17362423b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"
Sep 19 10:57:47.777742 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:47.777649    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0933555e-5813-45a1-bbbd-0bd17362423b-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz\" (UID: \"0933555e-5813-45a1-bbbd-0bd17362423b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"
Sep 19 10:57:47.782772 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:47.782750    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdww6\" (UniqueName: \"kubernetes.io/projected/0933555e-5813-45a1-bbbd-0bd17362423b-kube-api-access-gdww6\") pod \"maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz\" (UID: \"0933555e-5813-45a1-bbbd-0bd17362423b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"
Sep 19 10:57:47.836414 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:47.836385    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"
Sep 19 10:57:48.114633 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:48.114607    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"]
Sep 19 10:57:48.116788 ip-10-0-142-165 kubenswrapper[2576]: W0919 10:57:48.116758    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0933555e_5813_45a1_bbbd_0bd17362423b.slice/crio-52923eb4199294d544530bfbc6d339ca715cd6c990192a67a2b3fb9421dc67e9 WatchSource:0}: Error finding container 52923eb4199294d544530bfbc6d339ca715cd6c990192a67a2b3fb9421dc67e9: Status 404 returned error can't find the container with id 52923eb4199294d544530bfbc6d339ca715cd6c990192a67a2b3fb9421dc67e9
Sep 19 10:57:48.150311 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:48.150273    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz" event={"ID":"0933555e-5813-45a1-bbbd-0bd17362423b","Type":"ContainerStarted","Data":"52923eb4199294d544530bfbc6d339ca715cd6c990192a67a2b3fb9421dc67e9"}
Sep 19 10:57:49.157276 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:49.157230    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz" event={"ID":"0933555e-5813-45a1-bbbd-0bd17362423b","Type":"ContainerStarted","Data":"6c1f37bbb573e63a908aea9d3b4362e92bdaa58d584709011ee619bf5d699425"}
Sep 19 10:57:49.157765 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:49.157379    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"
Sep 19 10:57:49.183221 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:49.183164    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz" podStartSLOduration=1.787149804 podStartE2EDuration="2.183146285s" totalImagesPullingTime="395.996481ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:57:49.180633683 +0000 UTC m=+1278.676939773" watchObservedRunningTime="2026-09-19 10:57:49.183146285 +0000 UTC m=+1278.679452378"
Sep 19 10:57:55.168355 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:55.168326    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"
Sep 19 10:57:55.396522 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:57:55.396483    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"]
Sep 19 10:58:07.290970 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:07.290874    2576 generic.go:356] "Generic (PLEG): container finished" podID="5915405f-7676-421d-9230-b3c30b27a115" containerID="201b6f91475dbaf0514b38f1f4bf6f5b53552f9ea01cbf117c00c5742c63c6a4" exitCode=0
Sep 19 10:58:07.290970 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:07.290927    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd" event={"ID":"5915405f-7676-421d-9230-b3c30b27a115","Type":"ContainerDied","Data":"201b6f91475dbaf0514b38f1f4bf6f5b53552f9ea01cbf117c00c5742c63c6a4"}
Sep 19 10:58:07.291494 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:07.291310    2576 scope.go:117] "RemoveContainer" containerID="201b6f91475dbaf0514b38f1f4bf6f5b53552f9ea01cbf117c00c5742c63c6a4"
Sep 19 10:58:07.292490 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:07.292461    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:58:07.292561 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:07.292549    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:58:07.292604 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:07.292582    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 10:58:08.300576 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:08.300535    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd" event={"ID":"5915405f-7676-421d-9230-b3c30b27a115","Type":"ContainerStarted","Data":"feb837d77fce3cc330894319cecd49d7385d12150241403209380277fe375a57"}
Sep 19 10:58:08.973933 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:08.973898    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:58:08.975395 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:08.975364    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 19 10:58:08.975529 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:08.975416    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd" podUID="5915405f-7676-421d-9230-b3c30b27a115" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 19 10:58:09.974499 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:09.974462    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 19 10:58:09.974869 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:09.974525    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd" podUID="5915405f-7676-421d-9230-b3c30b27a115" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 19 10:58:10.416885 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.416838    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q" podUID="b8ac3298-0bff-4317-a562-34c73da187e8" containerName="main" containerID="cri-o://718de77f5a48d39e553029e83b37df3b6ffed2b80c3638a31cdf12e02a130a36" gracePeriod=45
Sep 19 10:58:10.723447 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.723419    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:58:10.816653 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.816627    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-model-cache\") pod \"b8ac3298-0bff-4317-a562-34c73da187e8\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") "
Sep 19 10:58:10.816826 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.816710    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8ac3298-0bff-4317-a562-34c73da187e8-tls-certs\") pod \"b8ac3298-0bff-4317-a562-34c73da187e8\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") "
Sep 19 10:58:10.816826 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.816731    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7hjp\" (UniqueName: \"kubernetes.io/projected/b8ac3298-0bff-4317-a562-34c73da187e8-kube-api-access-w7hjp\") pod \"b8ac3298-0bff-4317-a562-34c73da187e8\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") "
Sep 19 10:58:10.816826 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.816801    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-dshm\") pod \"b8ac3298-0bff-4317-a562-34c73da187e8\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") "
Sep 19 10:58:10.817002 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.816847    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-tmp-dir\") pod \"b8ac3298-0bff-4317-a562-34c73da187e8\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") "
Sep 19 10:58:10.817002 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.816872    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-home\") pod \"b8ac3298-0bff-4317-a562-34c73da187e8\" (UID: \"b8ac3298-0bff-4317-a562-34c73da187e8\") "
Sep 19 10:58:10.817002 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.816877    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-model-cache" (OuterVolumeSpecName: "model-cache") pod "b8ac3298-0bff-4317-a562-34c73da187e8" (UID: "b8ac3298-0bff-4317-a562-34c73da187e8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:58:10.817224 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.817094    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b8ac3298-0bff-4317-a562-34c73da187e8" (UID: "b8ac3298-0bff-4317-a562-34c73da187e8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:58:10.817224 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.817183    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-home" (OuterVolumeSpecName: "home") pod "b8ac3298-0bff-4317-a562-34c73da187e8" (UID: "b8ac3298-0bff-4317-a562-34c73da187e8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:58:10.817333 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.817300    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-tmp-dir\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:58:10.817333 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.817320    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-home\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:58:10.817616 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.817334    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-model-cache\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:58:10.819068 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.819017    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-dshm" (OuterVolumeSpecName: "dshm") pod "b8ac3298-0bff-4317-a562-34c73da187e8" (UID: "b8ac3298-0bff-4317-a562-34c73da187e8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:58:10.819187 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.819068    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8ac3298-0bff-4317-a562-34c73da187e8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b8ac3298-0bff-4317-a562-34c73da187e8" (UID: "b8ac3298-0bff-4317-a562-34c73da187e8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:58:10.819187 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.819145    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8ac3298-0bff-4317-a562-34c73da187e8-kube-api-access-w7hjp" (OuterVolumeSpecName: "kube-api-access-w7hjp") pod "b8ac3298-0bff-4317-a562-34c73da187e8" (UID: "b8ac3298-0bff-4317-a562-34c73da187e8"). InnerVolumeSpecName "kube-api-access-w7hjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:58:10.918463 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.918435    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8ac3298-0bff-4317-a562-34c73da187e8-dshm\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:58:10.918463 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.918457    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8ac3298-0bff-4317-a562-34c73da187e8-tls-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:58:10.918463 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.918466    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7hjp\" (UniqueName: \"kubernetes.io/projected/b8ac3298-0bff-4317-a562-34c73da187e8-kube-api-access-w7hjp\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 10:58:10.974265 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.974194    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 19 10:58:10.974265 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:10.974244    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd" podUID="5915405f-7676-421d-9230-b3c30b27a115" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 19 10:58:11.318645 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:11.318551    2576 generic.go:356] "Generic (PLEG): container finished" podID="b8ac3298-0bff-4317-a562-34c73da187e8" containerID="718de77f5a48d39e553029e83b37df3b6ffed2b80c3638a31cdf12e02a130a36" exitCode=0
Sep 19 10:58:11.318645 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:11.318593    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q" event={"ID":"b8ac3298-0bff-4317-a562-34c73da187e8","Type":"ContainerDied","Data":"718de77f5a48d39e553029e83b37df3b6ffed2b80c3638a31cdf12e02a130a36"}
Sep 19 10:58:11.318645 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:11.318629    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q" event={"ID":"b8ac3298-0bff-4317-a562-34c73da187e8","Type":"ContainerDied","Data":"8d0d2ed5bf74946b0e5e81e8bfa203de8b68ef65580f9b6a3cec52a594f106e1"}
Sep 19 10:58:11.318645 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:11.318642    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"
Sep 19 10:58:11.319288 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:11.318647    2576 scope.go:117] "RemoveContainer" containerID="718de77f5a48d39e553029e83b37df3b6ffed2b80c3638a31cdf12e02a130a36"
Sep 19 10:58:11.331626 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:11.331605    2576 scope.go:117] "RemoveContainer" containerID="718de77f5a48d39e553029e83b37df3b6ffed2b80c3638a31cdf12e02a130a36"
Sep 19 10:58:11.332311 ip-10-0-142-165 kubenswrapper[2576]: E0919 10:58:11.332282    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"718de77f5a48d39e553029e83b37df3b6ffed2b80c3638a31cdf12e02a130a36\": container with ID starting with 718de77f5a48d39e553029e83b37df3b6ffed2b80c3638a31cdf12e02a130a36 not found: ID does not exist" containerID="718de77f5a48d39e553029e83b37df3b6ffed2b80c3638a31cdf12e02a130a36"
Sep 19 10:58:11.332432 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:11.332322    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"718de77f5a48d39e553029e83b37df3b6ffed2b80c3638a31cdf12e02a130a36"} err="failed to get container status \"718de77f5a48d39e553029e83b37df3b6ffed2b80c3638a31cdf12e02a130a36\": rpc error: code = NotFound desc = could not find container \"718de77f5a48d39e553029e83b37df3b6ffed2b80c3638a31cdf12e02a130a36\": container with ID starting with 718de77f5a48d39e553029e83b37df3b6ffed2b80c3638a31cdf12e02a130a36 not found: ID does not exist"
Sep 19 10:58:11.352267 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:11.352239    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"]
Sep 19 10:58:11.353952 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:11.353919    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-5116ec/e2e-conflict-b-9f7696ca-kserve-758c4bf84-lmm5q"]
Sep 19 10:58:11.974232 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:11.974199    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 19 10:58:11.974424 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:11.974256    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd" podUID="5915405f-7676-421d-9230-b3c30b27a115" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 19 10:58:12.114159 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:12.114130    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:58:12.974496 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:12.974459    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 19 10:58:12.974901 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:12.974514    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd" podUID="5915405f-7676-421d-9230-b3c30b27a115" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 19 10:58:13.001895 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:13.001865    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8ac3298-0bff-4317-a562-34c73da187e8" path="/var/lib/kubelet/pods/b8ac3298-0bff-4317-a562-34c73da187e8/volumes"
Sep 19 10:58:13.978573 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:13.978544    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 10:58:13.979564 ip-10-0-142-165 kubenswrapper[2576]: I0919 10:58:13.979543    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 11:00:02.993138 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:02.993099    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"]
Sep 19 11:00:02.993861 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:02.993840    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8ac3298-0bff-4317-a562-34c73da187e8" containerName="main"
Sep 19 11:00:02.993960 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:02.993864    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8ac3298-0bff-4317-a562-34c73da187e8" containerName="main"
Sep 19 11:00:02.994024 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:02.994006    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8ac3298-0bff-4317-a562-34c73da187e8" containerName="main"
Sep 19 11:00:02.997170 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:02.997142    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.000130 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.000114    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-000aa3-openshift-default-dockercfg-mvrmp\""
Sep 19 11:00:03.009164 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.009145    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"]
Sep 19 11:00:03.054535 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.054509    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhjk5\" (UniqueName: \"kubernetes.io/projected/5d763461-6001-4ef2-9588-1b76b66a3cf4-kube-api-access-rhjk5\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.054660 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.054548    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-credential-socket\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.054660 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.054579    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-workload-socket\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.054748 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.054672    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-podinfo\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.054793 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.054768    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-token\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.054867 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.054843    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-envoy\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.054926 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.054902    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-workload-certs\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.054982 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.054959    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d763461-6001-4ef2-9588-1b76b66a3cf4-istiod-ca-cert\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.055033 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.055001    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-data\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.155844 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.155785    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-token\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.156078 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.155892    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-envoy\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.156078 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.155947    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-workload-certs\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.156078 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.156075    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d763461-6001-4ef2-9588-1b76b66a3cf4-istiod-ca-cert\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.156305 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.156107    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-data\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.156305 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.156138    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhjk5\" (UniqueName: \"kubernetes.io/projected/5d763461-6001-4ef2-9588-1b76b66a3cf4-kube-api-access-rhjk5\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.156305 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.156174    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-credential-socket\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.156305 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.156209    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-workload-socket\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.156305 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.156284    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-podinfo\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.156553 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.156405    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-workload-certs\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.156810 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.156784    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-credential-socket\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.156921 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.156863    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-data\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.157001 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.156973    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-workload-socket\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.157593 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.157423    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d763461-6001-4ef2-9588-1b76b66a3cf4-istiod-ca-cert\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.159022 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.158995    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-envoy\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.159365 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.159348    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-podinfo\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.163738 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.163711    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-token\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.163888 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.163871    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhjk5\" (UniqueName: \"kubernetes.io/projected/5d763461-6001-4ef2-9588-1b76b66a3cf4-kube-api-access-rhjk5\") pod \"e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") " pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.309366 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.309301    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:03.615478 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.615452    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"]
Sep 19 11:00:03.617128 ip-10-0-142-165 kubenswrapper[2576]: W0919 11:00:03.617100    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d763461_6001_4ef2_9588_1b76b66a3cf4.slice/crio-45a6bc909427e02fc9eb6d6c653d61febdaa6b11cb5dc9a20b95accba1c34ae2 WatchSource:0}: Error finding container 45a6bc909427e02fc9eb6d6c653d61febdaa6b11cb5dc9a20b95accba1c34ae2: Status 404 returned error can't find the container with id 45a6bc909427e02fc9eb6d6c653d61febdaa6b11cb5dc9a20b95accba1c34ae2
Sep 19 11:00:03.620146 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.620105    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:00:03.620228 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.620192    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:00:03.620268 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.620240    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:00:03.960248 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.960160    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw" event={"ID":"5d763461-6001-4ef2-9588-1b76b66a3cf4","Type":"ContainerStarted","Data":"88a3ecd70477e4c04c65a6076a880fc44343d83df016c0062792e8dc7cb991f5"}
Sep 19 11:00:03.960248 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.960220    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw" event={"ID":"5d763461-6001-4ef2-9588-1b76b66a3cf4","Type":"ContainerStarted","Data":"45a6bc909427e02fc9eb6d6c653d61febdaa6b11cb5dc9a20b95accba1c34ae2"}
Sep 19 11:00:03.980585 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:03.980531    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw" podStartSLOduration=1.980515185 podStartE2EDuration="1.980515185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:00:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:00:03.97737389 +0000 UTC m=+1413.473679984" watchObservedRunningTime="2026-09-19 11:00:03.980515185 +0000 UTC m=+1413.476821273"
Sep 19 11:00:04.310489 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:04.310416    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:04.315948 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:04.315920    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:04.966183 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:04.966148    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:04.967082 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:04.967032    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:35.968011 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:35.967495    2576 patch_prober.go:28] interesting pod/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 19 11:00:35.968011 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:35.967587    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw" podUID="5d763461-6001-4ef2-9588-1b76b66a3cf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 19 11:00:36.148980 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.148909    2576 generic.go:356] "Generic (PLEG): container finished" podID="5d763461-6001-4ef2-9588-1b76b66a3cf4" containerID="88a3ecd70477e4c04c65a6076a880fc44343d83df016c0062792e8dc7cb991f5" exitCode=0
Sep 19 11:00:36.149158 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.148980    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw" event={"ID":"5d763461-6001-4ef2-9588-1b76b66a3cf4","Type":"ContainerDied","Data":"88a3ecd70477e4c04c65a6076a880fc44343d83df016c0062792e8dc7cb991f5"}
Sep 19 11:00:36.149527 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.149513    2576 scope.go:117] "RemoveContainer" containerID="88a3ecd70477e4c04c65a6076a880fc44343d83df016c0062792e8dc7cb991f5"
Sep 19 11:00:36.150740 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.150712    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:00:36.150815 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.150778    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:00:36.150815 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.150809    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:00:36.769958 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.769923    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"]
Sep 19 11:00:36.773036 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.773014    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.775703 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.775680    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-0c8f26-openshift-default-dockercfg-pq59f\""
Sep 19 11:00:36.786009 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.785950    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"]
Sep 19 11:00:36.807571 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.807546    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-istio-envoy\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.807692 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.807601    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-workload-socket\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.807749 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.807692    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-workload-certs\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.807793 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.807760    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hksxn\" (UniqueName: \"kubernetes.io/projected/4a51a259-d565-4254-9176-dc7205fc61ac-kube-api-access-hksxn\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.808029 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.807829    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a51a259-d565-4254-9176-dc7205fc61ac-istio-podinfo\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.808029 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.807873    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-credential-socket\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.808029 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.807900    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a51a259-d565-4254-9176-dc7205fc61ac-istiod-ca-cert\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.808029 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.807956    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-istio-data\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.808029 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.808014    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a51a259-d565-4254-9176-dc7205fc61ac-istio-token\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.909532 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.909497    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hksxn\" (UniqueName: \"kubernetes.io/projected/4a51a259-d565-4254-9176-dc7205fc61ac-kube-api-access-hksxn\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.909532 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.909539    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a51a259-d565-4254-9176-dc7205fc61ac-istio-podinfo\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.909803 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.909556    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-credential-socket\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.909803 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.909572    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a51a259-d565-4254-9176-dc7205fc61ac-istiod-ca-cert\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.909803 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.909715    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-istio-data\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.909803 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.909787    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a51a259-d565-4254-9176-dc7205fc61ac-istio-token\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.910030 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.909862    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-istio-envoy\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.910030 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.909928    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-workload-socket\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.910030 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.909977    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-workload-certs\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.910030 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.909985    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-credential-socket\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.910273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.910158    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-istio-data\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.910399 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.910378    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-workload-socket\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.910458 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.910394    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-workload-certs\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.910458 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.910416    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a51a259-d565-4254-9176-dc7205fc61ac-istiod-ca-cert\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.912251 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.912232    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-istio-envoy\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.912339 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.912284    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a51a259-d565-4254-9176-dc7205fc61ac-istio-podinfo\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.917908 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.917881    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hksxn\" (UniqueName: \"kubernetes.io/projected/4a51a259-d565-4254-9176-dc7205fc61ac-kube-api-access-hksxn\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:36.918276 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:36.918260    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a51a259-d565-4254-9176-dc7205fc61ac-istio-token\") pod \"e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") " pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:37.086252 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:37.086221    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:37.175798 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:37.174323    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw" event={"ID":"5d763461-6001-4ef2-9588-1b76b66a3cf4","Type":"ContainerStarted","Data":"171f8ca10ff2e805efeab74dbae0c420470249b05419ef531191e2efbcbb0319"}
Sep 19 11:00:37.309765 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:37.309729    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:37.309965 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:37.309916    2576 patch_prober.go:28] interesting pod/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 19 11:00:37.310081 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:37.309969    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw" podUID="5d763461-6001-4ef2-9588-1b76b66a3cf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 19 11:00:37.389517 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:37.389493    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"]
Sep 19 11:00:37.391182 ip-10-0-142-165 kubenswrapper[2576]: W0919 11:00:37.391151    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a51a259_d565_4254_9176_dc7205fc61ac.slice/crio-6e385c44029d8c840300f467de76b5abc77f5cdccab0863545562aa412b0a997 WatchSource:0}: Error finding container 6e385c44029d8c840300f467de76b5abc77f5cdccab0863545562aa412b0a997: Status 404 returned error can't find the container with id 6e385c44029d8c840300f467de76b5abc77f5cdccab0863545562aa412b0a997
Sep 19 11:00:37.394235 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:37.394194    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:00:37.394342 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:37.394285    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:00:37.394342 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:37.394332    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:00:38.185717 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:38.185677    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99" event={"ID":"4a51a259-d565-4254-9176-dc7205fc61ac","Type":"ContainerStarted","Data":"a6253ab6b2bfc7b65ded5909cfdade19a682110d6b1255bf6ea358ddbc218dd3"}
Sep 19 11:00:38.186119 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:38.185742    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99" event={"ID":"4a51a259-d565-4254-9176-dc7205fc61ac","Type":"ContainerStarted","Data":"6e385c44029d8c840300f467de76b5abc77f5cdccab0863545562aa412b0a997"}
Sep 19 11:00:38.204148 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:38.204098    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99" podStartSLOduration=2.20408308 podStartE2EDuration="2.20408308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:00:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:00:38.202903356 +0000 UTC m=+1447.699209471" watchObservedRunningTime="2026-09-19 11:00:38.20408308 +0000 UTC m=+1447.700389167"
Sep 19 11:00:38.310401 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:38.310369    2576 patch_prober.go:28] interesting pod/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 19 11:00:38.310605 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:38.310420    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw" podUID="5d763461-6001-4ef2-9588-1b76b66a3cf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 19 11:00:39.086945 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:39.086912    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:39.092426 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:39.092400    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:39.192095 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:39.192037    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:39.193160 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:39.193141    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:00:39.310266 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:39.310229    2576 patch_prober.go:28] interesting pod/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 19 11:00:39.310455 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:39.310285    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw" podUID="5d763461-6001-4ef2-9588-1b76b66a3cf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 19 11:00:40.309927 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:40.309892    2576 patch_prober.go:28] interesting pod/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 19 11:00:40.310344 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:40.309951    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw" podUID="5d763461-6001-4ef2-9588-1b76b66a3cf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 19 11:00:41.310205 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:41.310171    2576 patch_prober.go:28] interesting pod/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 19 11:00:41.310572 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:41.310220    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw" podUID="5d763461-6001-4ef2-9588-1b76b66a3cf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 19 11:00:42.340674 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:42.340643    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:42.341137 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:42.341082    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:42.343624 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:42.343603    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:00:45.529718 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:45.529676    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"]
Sep 19 11:00:45.558820 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:45.558784    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"
Sep 19 11:00:45.562148 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:45.562124    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-0c8f26-serving-cert\""
Sep 19 11:00:45.566026 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:45.565991    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"]
Sep 19 11:00:45.621929 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:45.621901    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvlsp\" (UniqueName: \"kubernetes.io/projected/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-kube-api-access-rvlsp\") pod \"maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p\" (UID: \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"
Sep 19 11:00:45.622121 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:45.621952    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-maas-api-tls\") pod \"maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p\" (UID: \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"
Sep 19 11:00:45.622121 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:45.621967    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-openshift-service-ca\") pod \"maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p\" (UID: \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"
Sep 19 11:00:45.622226 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:45.622148    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-metrics-certs\") pod \"maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p\" (UID: \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"
Sep 19 11:00:45.723613 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:45.723582    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-metrics-certs\") pod \"maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p\" (UID: \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"
Sep 19 11:00:45.723768 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:45.723622    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvlsp\" (UniqueName: \"kubernetes.io/projected/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-kube-api-access-rvlsp\") pod \"maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p\" (UID: \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"
Sep 19 11:00:45.723768 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:45.723660    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-maas-api-tls\") pod \"maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p\" (UID: \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"
Sep 19 11:00:45.723768 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:45.723701    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-openshift-service-ca\") pod \"maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p\" (UID: \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"
Sep 19 11:00:45.724331 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:45.724306    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-openshift-service-ca\") pod \"maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p\" (UID: \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"
Sep 19 11:00:45.726163 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:45.726134    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-metrics-certs\") pod \"maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p\" (UID: \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"
Sep 19 11:00:45.726163 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:45.726149    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-maas-api-tls\") pod \"maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p\" (UID: \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"
Sep 19 11:00:45.731221 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:45.731198    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvlsp\" (UniqueName: \"kubernetes.io/projected/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-kube-api-access-rvlsp\") pod \"maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p\" (UID: \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"
Sep 19 11:00:45.882155 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:45.882117    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"
Sep 19 11:00:46.162864 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:46.162828    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"]
Sep 19 11:00:46.165055 ip-10-0-142-165 kubenswrapper[2576]: W0919 11:00:46.165000    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode94ce1b3_43d9_4712_902d_a60ba1ff5e6b.slice/crio-7e830ec7e88dc335a210a6f4f714fc29e71249f7ab0815e6a129cf9f034252b9 WatchSource:0}: Error finding container 7e830ec7e88dc335a210a6f4f714fc29e71249f7ab0815e6a129cf9f034252b9: Status 404 returned error can't find the container with id 7e830ec7e88dc335a210a6f4f714fc29e71249f7ab0815e6a129cf9f034252b9
Sep 19 11:00:46.166945 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:46.166917    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 11:00:46.234648 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:46.234603    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p" event={"ID":"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b","Type":"ContainerStarted","Data":"7e830ec7e88dc335a210a6f4f714fc29e71249f7ab0815e6a129cf9f034252b9"}
Sep 19 11:00:47.242275 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:47.242244    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p" event={"ID":"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b","Type":"ContainerStarted","Data":"372d3c7d33c9c921e743a88c7a8f154cc13f2bb41c598e26bc58d28d5305e017"}
Sep 19 11:00:47.242741 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:47.242299    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"
Sep 19 11:00:47.261119 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:47.261074    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p" podStartSLOduration=1.919664487 podStartE2EDuration="2.261058116s" totalImagesPullingTime="341.393629ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:00:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:00:47.25796137 +0000 UTC m=+1456.754267475" watchObservedRunningTime="2026-09-19 11:00:47.261058116 +0000 UTC m=+1456.757364199"
Sep 19 11:00:53.252855 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:00:53.252828    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"
Sep 19 11:01:19.452768 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:19.452694    2576 generic.go:356] "Generic (PLEG): container finished" podID="4a51a259-d565-4254-9176-dc7205fc61ac" containerID="a6253ab6b2bfc7b65ded5909cfdade19a682110d6b1255bf6ea358ddbc218dd3" exitCode=0
Sep 19 11:01:19.453161 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:19.452775    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99" event={"ID":"4a51a259-d565-4254-9176-dc7205fc61ac","Type":"ContainerDied","Data":"a6253ab6b2bfc7b65ded5909cfdade19a682110d6b1255bf6ea358ddbc218dd3"}
Sep 19 11:01:19.453240 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:19.453224    2576 scope.go:117] "RemoveContainer" containerID="a6253ab6b2bfc7b65ded5909cfdade19a682110d6b1255bf6ea358ddbc218dd3"
Sep 19 11:01:19.454488 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:19.454462    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:01:19.454559 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:19.454531    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:01:19.454595 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:19.454563    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:01:20.461118 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:20.461078    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99" event={"ID":"4a51a259-d565-4254-9176-dc7205fc61ac","Type":"ContainerStarted","Data":"a9a758e060cd1673082f2a7f68c4487bdb4b495885d2b1352f7b1948fc929066"}
Sep 19 11:01:21.086654 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:21.086629    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:01:21.087725 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:21.087702    2576 patch_prober.go:28] interesting pod/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 19 11:01:21.087848 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:21.087745    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99" podUID="4a51a259-d565-4254-9176-dc7205fc61ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 19 11:01:22.086620 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:22.086579    2576 patch_prober.go:28] interesting pod/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 19 11:01:22.087009 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:22.086647    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99" podUID="4a51a259-d565-4254-9176-dc7205fc61ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 19 11:01:23.087307 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:23.087272    2576 patch_prober.go:28] interesting pod/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 19 11:01:23.087684 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:23.087323    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99" podUID="4a51a259-d565-4254-9176-dc7205fc61ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 19 11:01:24.087008 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:24.086969    2576 patch_prober.go:28] interesting pod/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 19 11:01:24.087221 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:24.087029    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99" podUID="4a51a259-d565-4254-9176-dc7205fc61ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 19 11:01:24.192290 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:24.192260    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:01:25.087066 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:25.087015    2576 patch_prober.go:28] interesting pod/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 19 11:01:25.087314 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:25.087100    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99" podUID="4a51a259-d565-4254-9176-dc7205fc61ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 19 11:01:26.086711 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:26.086679    2576 patch_prober.go:28] interesting pod/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 19 11:01:26.087141 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:26.086736    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99" podUID="4a51a259-d565-4254-9176-dc7205fc61ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 19 11:01:27.090884 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:27.090857    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:01:27.091872 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:27.091855    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:01:31.042745 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:31.042717    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 11:01:31.048906 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:31.048885    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 11:01:35.528516 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:35.528484    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"]
Sep 19 11:01:35.528919 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:35.528700    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw" podUID="5d763461-6001-4ef2-9588-1b76b66a3cf4" containerName="istio-proxy" containerID="cri-o://171f8ca10ff2e805efeab74dbae0c420470249b05419ef531191e2efbcbb0319" gracePeriod=30
Sep 19 11:01:40.798910 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.798854    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:01:40.992867 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.992842    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-workload-socket\") pod \"5d763461-6001-4ef2-9588-1b76b66a3cf4\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") "
Sep 19 11:01:40.993157 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.992879    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhjk5\" (UniqueName: \"kubernetes.io/projected/5d763461-6001-4ef2-9588-1b76b66a3cf4-kube-api-access-rhjk5\") pod \"5d763461-6001-4ef2-9588-1b76b66a3cf4\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") "
Sep 19 11:01:40.993157 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.992923    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-data\") pod \"5d763461-6001-4ef2-9588-1b76b66a3cf4\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") "
Sep 19 11:01:40.993157 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.992962    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-credential-socket\") pod \"5d763461-6001-4ef2-9588-1b76b66a3cf4\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") "
Sep 19 11:01:40.993157 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.992989    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-podinfo\") pod \"5d763461-6001-4ef2-9588-1b76b66a3cf4\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") "
Sep 19 11:01:40.993157 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.993011    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-workload-certs\") pod \"5d763461-6001-4ef2-9588-1b76b66a3cf4\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") "
Sep 19 11:01:40.993157 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.993071    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d763461-6001-4ef2-9588-1b76b66a3cf4-istiod-ca-cert\") pod \"5d763461-6001-4ef2-9588-1b76b66a3cf4\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") "
Sep 19 11:01:40.993157 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.993099    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-envoy\") pod \"5d763461-6001-4ef2-9588-1b76b66a3cf4\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") "
Sep 19 11:01:40.993157 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.993149    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-token\") pod \"5d763461-6001-4ef2-9588-1b76b66a3cf4\" (UID: \"5d763461-6001-4ef2-9588-1b76b66a3cf4\") "
Sep 19 11:01:40.993574 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.993154    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5d763461-6001-4ef2-9588-1b76b66a3cf4" (UID: "5d763461-6001-4ef2-9588-1b76b66a3cf4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:01:40.993574 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.993303    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-data" (OuterVolumeSpecName: "istio-data") pod "5d763461-6001-4ef2-9588-1b76b66a3cf4" (UID: "5d763461-6001-4ef2-9588-1b76b66a3cf4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:01:40.993574 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.993321    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5d763461-6001-4ef2-9588-1b76b66a3cf4" (UID: "5d763461-6001-4ef2-9588-1b76b66a3cf4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:01:40.993574 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.993423    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5d763461-6001-4ef2-9588-1b76b66a3cf4" (UID: "5d763461-6001-4ef2-9588-1b76b66a3cf4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:01:40.993574 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.993475    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d763461-6001-4ef2-9588-1b76b66a3cf4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5d763461-6001-4ef2-9588-1b76b66a3cf4" (UID: "5d763461-6001-4ef2-9588-1b76b66a3cf4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:01:40.993769 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.993623    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:01:40.993769 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.993643    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:01:40.993769 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.993656    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:01:40.993769 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.993669    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:01:40.993769 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.993681    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d763461-6001-4ef2-9588-1b76b66a3cf4-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:01:40.995501 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.995468    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d763461-6001-4ef2-9588-1b76b66a3cf4-kube-api-access-rhjk5" (OuterVolumeSpecName: "kube-api-access-rhjk5") pod "5d763461-6001-4ef2-9588-1b76b66a3cf4" (UID: "5d763461-6001-4ef2-9588-1b76b66a3cf4"). InnerVolumeSpecName "kube-api-access-rhjk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:01:40.995635 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.995514    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5d763461-6001-4ef2-9588-1b76b66a3cf4" (UID: "5d763461-6001-4ef2-9588-1b76b66a3cf4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 11:01:40.995995 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.995962    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5d763461-6001-4ef2-9588-1b76b66a3cf4" (UID: "5d763461-6001-4ef2-9588-1b76b66a3cf4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:01:40.996165 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:40.996144    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-token" (OuterVolumeSpecName: "istio-token") pod "5d763461-6001-4ef2-9588-1b76b66a3cf4" (UID: "5d763461-6001-4ef2-9588-1b76b66a3cf4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:01:41.094705 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:41.094684    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:01:41.094705 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:41.094706    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:01:41.094837 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:41.094716    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhjk5\" (UniqueName: \"kubernetes.io/projected/5d763461-6001-4ef2-9588-1b76b66a3cf4-kube-api-access-rhjk5\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:01:41.094837 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:41.094724    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d763461-6001-4ef2-9588-1b76b66a3cf4-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:01:41.587676 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:41.587638    2576 generic.go:356] "Generic (PLEG): container finished" podID="5d763461-6001-4ef2-9588-1b76b66a3cf4" containerID="171f8ca10ff2e805efeab74dbae0c420470249b05419ef531191e2efbcbb0319" exitCode=0
Sep 19 11:01:41.587907 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:41.587689    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"
Sep 19 11:01:41.587907 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:41.587698    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw" event={"ID":"5d763461-6001-4ef2-9588-1b76b66a3cf4","Type":"ContainerDied","Data":"171f8ca10ff2e805efeab74dbae0c420470249b05419ef531191e2efbcbb0319"}
Sep 19 11:01:41.587907 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:41.587745    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw" event={"ID":"5d763461-6001-4ef2-9588-1b76b66a3cf4","Type":"ContainerDied","Data":"45a6bc909427e02fc9eb6d6c653d61febdaa6b11cb5dc9a20b95accba1c34ae2"}
Sep 19 11:01:41.587907 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:41.587767    2576 scope.go:117] "RemoveContainer" containerID="171f8ca10ff2e805efeab74dbae0c420470249b05419ef531191e2efbcbb0319"
Sep 19 11:01:41.597836 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:41.597821    2576 scope.go:117] "RemoveContainer" containerID="88a3ecd70477e4c04c65a6076a880fc44343d83df016c0062792e8dc7cb991f5"
Sep 19 11:01:41.606940 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:41.606917    2576 scope.go:117] "RemoveContainer" containerID="171f8ca10ff2e805efeab74dbae0c420470249b05419ef531191e2efbcbb0319"
Sep 19 11:01:41.607303 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:01:41.607277    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"171f8ca10ff2e805efeab74dbae0c420470249b05419ef531191e2efbcbb0319\": container with ID starting with 171f8ca10ff2e805efeab74dbae0c420470249b05419ef531191e2efbcbb0319 not found: ID does not exist" containerID="171f8ca10ff2e805efeab74dbae0c420470249b05419ef531191e2efbcbb0319"
Sep 19 11:01:41.607421 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:41.607306    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"171f8ca10ff2e805efeab74dbae0c420470249b05419ef531191e2efbcbb0319"} err="failed to get container status \"171f8ca10ff2e805efeab74dbae0c420470249b05419ef531191e2efbcbb0319\": rpc error: code = NotFound desc = could not find container \"171f8ca10ff2e805efeab74dbae0c420470249b05419ef531191e2efbcbb0319\": container with ID starting with 171f8ca10ff2e805efeab74dbae0c420470249b05419ef531191e2efbcbb0319 not found: ID does not exist"
Sep 19 11:01:41.607421 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:41.607323    2576 scope.go:117] "RemoveContainer" containerID="88a3ecd70477e4c04c65a6076a880fc44343d83df016c0062792e8dc7cb991f5"
Sep 19 11:01:41.607584 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:01:41.607568    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88a3ecd70477e4c04c65a6076a880fc44343d83df016c0062792e8dc7cb991f5\": container with ID starting with 88a3ecd70477e4c04c65a6076a880fc44343d83df016c0062792e8dc7cb991f5 not found: ID does not exist" containerID="88a3ecd70477e4c04c65a6076a880fc44343d83df016c0062792e8dc7cb991f5"
Sep 19 11:01:41.607627 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:41.607588    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88a3ecd70477e4c04c65a6076a880fc44343d83df016c0062792e8dc7cb991f5"} err="failed to get container status \"88a3ecd70477e4c04c65a6076a880fc44343d83df016c0062792e8dc7cb991f5\": rpc error: code = NotFound desc = could not find container \"88a3ecd70477e4c04c65a6076a880fc44343d83df016c0062792e8dc7cb991f5\": container with ID starting with 88a3ecd70477e4c04c65a6076a880fc44343d83df016c0062792e8dc7cb991f5 not found: ID does not exist"
Sep 19 11:01:41.609272 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:41.609253    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"]
Sep 19 11:01:41.612751 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:41.612731    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-000aa3-openshift-default-6d7478466b-sfmnw"]
Sep 19 11:01:43.002532 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:43.002503    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d763461-6001-4ef2-9588-1b76b66a3cf4" path="/var/lib/kubelet/pods/5d763461-6001-4ef2-9588-1b76b66a3cf4/volumes"
Sep 19 11:01:51.367779 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.367742    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"]
Sep 19 11:01:51.368312 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.367999    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p" podUID="e94ce1b3-43d9-4712-902d-a60ba1ff5e6b" containerName="maas-api" containerID="cri-o://372d3c7d33c9c921e743a88c7a8f154cc13f2bb41c598e26bc58d28d5305e017" gracePeriod=30
Sep 19 11:01:51.629761 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.629709    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"
Sep 19 11:01:51.652622 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.652469    2576 generic.go:356] "Generic (PLEG): container finished" podID="e94ce1b3-43d9-4712-902d-a60ba1ff5e6b" containerID="372d3c7d33c9c921e743a88c7a8f154cc13f2bb41c598e26bc58d28d5305e017" exitCode=0
Sep 19 11:01:51.652622 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.652518    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"
Sep 19 11:01:51.652622 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.652562    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p" event={"ID":"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b","Type":"ContainerDied","Data":"372d3c7d33c9c921e743a88c7a8f154cc13f2bb41c598e26bc58d28d5305e017"}
Sep 19 11:01:51.652875 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.652647    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p" event={"ID":"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b","Type":"ContainerDied","Data":"7e830ec7e88dc335a210a6f4f714fc29e71249f7ab0815e6a129cf9f034252b9"}
Sep 19 11:01:51.652875 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.652702    2576 scope.go:117] "RemoveContainer" containerID="372d3c7d33c9c921e743a88c7a8f154cc13f2bb41c598e26bc58d28d5305e017"
Sep 19 11:01:51.664883 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.664864    2576 scope.go:117] "RemoveContainer" containerID="372d3c7d33c9c921e743a88c7a8f154cc13f2bb41c598e26bc58d28d5305e017"
Sep 19 11:01:51.665256 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:01:51.665233    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"372d3c7d33c9c921e743a88c7a8f154cc13f2bb41c598e26bc58d28d5305e017\": container with ID starting with 372d3c7d33c9c921e743a88c7a8f154cc13f2bb41c598e26bc58d28d5305e017 not found: ID does not exist" containerID="372d3c7d33c9c921e743a88c7a8f154cc13f2bb41c598e26bc58d28d5305e017"
Sep 19 11:01:51.665325 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.665266    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"372d3c7d33c9c921e743a88c7a8f154cc13f2bb41c598e26bc58d28d5305e017"} err="failed to get container status \"372d3c7d33c9c921e743a88c7a8f154cc13f2bb41c598e26bc58d28d5305e017\": rpc error: code = NotFound desc = could not find container \"372d3c7d33c9c921e743a88c7a8f154cc13f2bb41c598e26bc58d28d5305e017\": container with ID starting with 372d3c7d33c9c921e743a88c7a8f154cc13f2bb41c598e26bc58d28d5305e017 not found: ID does not exist"
Sep 19 11:01:51.697558 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.697539    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-maas-api-tls\") pod \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\" (UID: \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\") "
Sep 19 11:01:51.697656 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.697595    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvlsp\" (UniqueName: \"kubernetes.io/projected/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-kube-api-access-rvlsp\") pod \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\" (UID: \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\") "
Sep 19 11:01:51.697700 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.697682    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-openshift-service-ca\") pod \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\" (UID: \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\") "
Sep 19 11:01:51.697779 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.697758    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-metrics-certs\") pod \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\" (UID: \"e94ce1b3-43d9-4712-902d-a60ba1ff5e6b\") "
Sep 19 11:01:51.698095 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.698071    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e94ce1b3-43d9-4712-902d-a60ba1ff5e6b" (UID: "e94ce1b3-43d9-4712-902d-a60ba1ff5e6b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:01:51.698189 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.698161    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-openshift-service-ca\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:01:51.699884 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.699859    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e94ce1b3-43d9-4712-902d-a60ba1ff5e6b" (UID: "e94ce1b3-43d9-4712-902d-a60ba1ff5e6b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:01:51.700006 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.699986    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-kube-api-access-rvlsp" (OuterVolumeSpecName: "kube-api-access-rvlsp") pod "e94ce1b3-43d9-4712-902d-a60ba1ff5e6b" (UID: "e94ce1b3-43d9-4712-902d-a60ba1ff5e6b"). InnerVolumeSpecName "kube-api-access-rvlsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:01:51.700081 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.700002    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e94ce1b3-43d9-4712-902d-a60ba1ff5e6b" (UID: "e94ce1b3-43d9-4712-902d-a60ba1ff5e6b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:01:51.799097 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.799026    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-metrics-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:01:51.799291 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.799104    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-maas-api-tls\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:01:51.799291 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.799120    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvlsp\" (UniqueName: \"kubernetes.io/projected/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b-kube-api-access-rvlsp\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:01:51.976389 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.976360    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"]
Sep 19 11:01:51.980474 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:51.980449    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0c8f26-5c7c8dcd79-bhm5p"]
Sep 19 11:01:53.002575 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:53.002541    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e94ce1b3-43d9-4712-902d-a60ba1ff5e6b" path="/var/lib/kubelet/pods/e94ce1b3-43d9-4712-902d-a60ba1ff5e6b/volumes"
Sep 19 11:01:58.085112 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:58.085080    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"]
Sep 19 11:01:58.085563 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:01:58.085344    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99" podUID="4a51a259-d565-4254-9176-dc7205fc61ac" containerName="istio-proxy" containerID="cri-o://a9a758e060cd1673082f2a7f68c4487bdb4b495885d2b1352f7b1948fc929066" gracePeriod=30
Sep 19 11:02:03.348811 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.348748    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:02:03.517065 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.517016    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-workload-certs\") pod \"4a51a259-d565-4254-9176-dc7205fc61ac\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") "
Sep 19 11:02:03.517206 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.517134    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a51a259-d565-4254-9176-dc7205fc61ac-istiod-ca-cert\") pod \"4a51a259-d565-4254-9176-dc7205fc61ac\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") "
Sep 19 11:02:03.517206 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.517164    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a51a259-d565-4254-9176-dc7205fc61ac-istio-token\") pod \"4a51a259-d565-4254-9176-dc7205fc61ac\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") "
Sep 19 11:02:03.517386 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.517214    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-workload-socket\") pod \"4a51a259-d565-4254-9176-dc7205fc61ac\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") "
Sep 19 11:02:03.517386 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.517256    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hksxn\" (UniqueName: \"kubernetes.io/projected/4a51a259-d565-4254-9176-dc7205fc61ac-kube-api-access-hksxn\") pod \"4a51a259-d565-4254-9176-dc7205fc61ac\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") "
Sep 19 11:02:03.517386 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.517294    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-istio-data\") pod \"4a51a259-d565-4254-9176-dc7205fc61ac\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") "
Sep 19 11:02:03.517386 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.517317    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4a51a259-d565-4254-9176-dc7205fc61ac" (UID: "4a51a259-d565-4254-9176-dc7205fc61ac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:02:03.517386 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.517377    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a51a259-d565-4254-9176-dc7205fc61ac-istio-podinfo\") pod \"4a51a259-d565-4254-9176-dc7205fc61ac\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") "
Sep 19 11:02:03.517642 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.517402    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-istio-envoy\") pod \"4a51a259-d565-4254-9176-dc7205fc61ac\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") "
Sep 19 11:02:03.517642 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.517450    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-credential-socket\") pod \"4a51a259-d565-4254-9176-dc7205fc61ac\" (UID: \"4a51a259-d565-4254-9176-dc7205fc61ac\") "
Sep 19 11:02:03.517642 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.517563    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a51a259-d565-4254-9176-dc7205fc61ac-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4a51a259-d565-4254-9176-dc7205fc61ac" (UID: "4a51a259-d565-4254-9176-dc7205fc61ac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:02:03.517789 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.517670    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-istio-data" (OuterVolumeSpecName: "istio-data") pod "4a51a259-d565-4254-9176-dc7205fc61ac" (UID: "4a51a259-d565-4254-9176-dc7205fc61ac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:02:03.517887 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.517863    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4a51a259-d565-4254-9176-dc7205fc61ac" (UID: "4a51a259-d565-4254-9176-dc7205fc61ac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:02:03.518312 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.518248    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4a51a259-d565-4254-9176-dc7205fc61ac" (UID: "4a51a259-d565-4254-9176-dc7205fc61ac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:02:03.518497 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.518479    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a51a259-d565-4254-9176-dc7205fc61ac-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:03.518551 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.518504    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:03.518551 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.518520    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:03.518551 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.518536    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:03.518551 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.518548    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:03.519731 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.519700    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a51a259-d565-4254-9176-dc7205fc61ac-istio-token" (OuterVolumeSpecName: "istio-token") pod "4a51a259-d565-4254-9176-dc7205fc61ac" (UID: "4a51a259-d565-4254-9176-dc7205fc61ac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:02:03.520007 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.519986    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4a51a259-d565-4254-9176-dc7205fc61ac-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4a51a259-d565-4254-9176-dc7205fc61ac" (UID: "4a51a259-d565-4254-9176-dc7205fc61ac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 11:02:03.520218 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.520198    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4a51a259-d565-4254-9176-dc7205fc61ac" (UID: "4a51a259-d565-4254-9176-dc7205fc61ac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:02:03.520528 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.520509    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a51a259-d565-4254-9176-dc7205fc61ac-kube-api-access-hksxn" (OuterVolumeSpecName: "kube-api-access-hksxn") pod "4a51a259-d565-4254-9176-dc7205fc61ac" (UID: "4a51a259-d565-4254-9176-dc7205fc61ac"). InnerVolumeSpecName "kube-api-access-hksxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:02:03.619948 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.619889    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hksxn\" (UniqueName: \"kubernetes.io/projected/4a51a259-d565-4254-9176-dc7205fc61ac-kube-api-access-hksxn\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:03.619948 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.619914    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a51a259-d565-4254-9176-dc7205fc61ac-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:03.619948 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.619924    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a51a259-d565-4254-9176-dc7205fc61ac-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:03.619948 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.619932    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a51a259-d565-4254-9176-dc7205fc61ac-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:03.729468 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.729438    2576 generic.go:356] "Generic (PLEG): container finished" podID="4a51a259-d565-4254-9176-dc7205fc61ac" containerID="a9a758e060cd1673082f2a7f68c4487bdb4b495885d2b1352f7b1948fc929066" exitCode=0
Sep 19 11:02:03.729618 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.729504    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"
Sep 19 11:02:03.729618 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.729517    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99" event={"ID":"4a51a259-d565-4254-9176-dc7205fc61ac","Type":"ContainerDied","Data":"a9a758e060cd1673082f2a7f68c4487bdb4b495885d2b1352f7b1948fc929066"}
Sep 19 11:02:03.729618 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.729576    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99" event={"ID":"4a51a259-d565-4254-9176-dc7205fc61ac","Type":"ContainerDied","Data":"6e385c44029d8c840300f467de76b5abc77f5cdccab0863545562aa412b0a997"}
Sep 19 11:02:03.729618 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.729599    2576 scope.go:117] "RemoveContainer" containerID="a9a758e060cd1673082f2a7f68c4487bdb4b495885d2b1352f7b1948fc929066"
Sep 19 11:02:03.739742 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.739731    2576 scope.go:117] "RemoveContainer" containerID="a6253ab6b2bfc7b65ded5909cfdade19a682110d6b1255bf6ea358ddbc218dd3"
Sep 19 11:02:03.748946 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.748928    2576 scope.go:117] "RemoveContainer" containerID="a9a758e060cd1673082f2a7f68c4487bdb4b495885d2b1352f7b1948fc929066"
Sep 19 11:02:03.749240 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:02:03.749220    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9a758e060cd1673082f2a7f68c4487bdb4b495885d2b1352f7b1948fc929066\": container with ID starting with a9a758e060cd1673082f2a7f68c4487bdb4b495885d2b1352f7b1948fc929066 not found: ID does not exist" containerID="a9a758e060cd1673082f2a7f68c4487bdb4b495885d2b1352f7b1948fc929066"
Sep 19 11:02:03.749328 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.749251    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9a758e060cd1673082f2a7f68c4487bdb4b495885d2b1352f7b1948fc929066"} err="failed to get container status \"a9a758e060cd1673082f2a7f68c4487bdb4b495885d2b1352f7b1948fc929066\": rpc error: code = NotFound desc = could not find container \"a9a758e060cd1673082f2a7f68c4487bdb4b495885d2b1352f7b1948fc929066\": container with ID starting with a9a758e060cd1673082f2a7f68c4487bdb4b495885d2b1352f7b1948fc929066 not found: ID does not exist"
Sep 19 11:02:03.749328 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.749274    2576 scope.go:117] "RemoveContainer" containerID="a6253ab6b2bfc7b65ded5909cfdade19a682110d6b1255bf6ea358ddbc218dd3"
Sep 19 11:02:03.749555 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:02:03.749539    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6253ab6b2bfc7b65ded5909cfdade19a682110d6b1255bf6ea358ddbc218dd3\": container with ID starting with a6253ab6b2bfc7b65ded5909cfdade19a682110d6b1255bf6ea358ddbc218dd3 not found: ID does not exist" containerID="a6253ab6b2bfc7b65ded5909cfdade19a682110d6b1255bf6ea358ddbc218dd3"
Sep 19 11:02:03.749600 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.749560    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6253ab6b2bfc7b65ded5909cfdade19a682110d6b1255bf6ea358ddbc218dd3"} err="failed to get container status \"a6253ab6b2bfc7b65ded5909cfdade19a682110d6b1255bf6ea358ddbc218dd3\": rpc error: code = NotFound desc = could not find container \"a6253ab6b2bfc7b65ded5909cfdade19a682110d6b1255bf6ea358ddbc218dd3\": container with ID starting with a6253ab6b2bfc7b65ded5909cfdade19a682110d6b1255bf6ea358ddbc218dd3 not found: ID does not exist"
Sep 19 11:02:03.751563 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.751543    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"]
Sep 19 11:02:03.755756 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:03.755736    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-0c8f26-openshift-default-7769d77968-2sk99"]
Sep 19 11:02:05.001457 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:05.001426    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a51a259-d565-4254-9176-dc7205fc61ac" path="/var/lib/kubelet/pods/4a51a259-d565-4254-9176-dc7205fc61ac/volumes"
Sep 19 11:02:08.909408 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:08.909379    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"]
Sep 19 11:02:08.909849 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:08.909635    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr" podUID="fcce4857-4551-4b82-8cd1-e8d2438c7699" containerName="maas-api" containerID="cri-o://278345ce84ea4ba22fc18e4d051a712290acada409198fc7607aed9d50ae5cce" gracePeriod=30
Sep 19 11:02:09.176841 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.176771    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"
Sep 19 11:02:09.275464 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.275430    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcce4857-4551-4b82-8cd1-e8d2438c7699-maas-api-tls\") pod \"fcce4857-4551-4b82-8cd1-e8d2438c7699\" (UID: \"fcce4857-4551-4b82-8cd1-e8d2438c7699\") "
Sep 19 11:02:09.275464 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.275467    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qg92\" (UniqueName: \"kubernetes.io/projected/fcce4857-4551-4b82-8cd1-e8d2438c7699-kube-api-access-6qg92\") pod \"fcce4857-4551-4b82-8cd1-e8d2438c7699\" (UID: \"fcce4857-4551-4b82-8cd1-e8d2438c7699\") "
Sep 19 11:02:09.275699 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.275626    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcce4857-4551-4b82-8cd1-e8d2438c7699-metrics-certs\") pod \"fcce4857-4551-4b82-8cd1-e8d2438c7699\" (UID: \"fcce4857-4551-4b82-8cd1-e8d2438c7699\") "
Sep 19 11:02:09.275748 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.275718    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcce4857-4551-4b82-8cd1-e8d2438c7699-openshift-service-ca\") pod \"fcce4857-4551-4b82-8cd1-e8d2438c7699\" (UID: \"fcce4857-4551-4b82-8cd1-e8d2438c7699\") "
Sep 19 11:02:09.276110 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.276077    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcce4857-4551-4b82-8cd1-e8d2438c7699-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fcce4857-4551-4b82-8cd1-e8d2438c7699" (UID: "fcce4857-4551-4b82-8cd1-e8d2438c7699"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:02:09.277932 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.277904    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcce4857-4551-4b82-8cd1-e8d2438c7699-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fcce4857-4551-4b82-8cd1-e8d2438c7699" (UID: "fcce4857-4551-4b82-8cd1-e8d2438c7699"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:02:09.278293 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.278269    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcce4857-4551-4b82-8cd1-e8d2438c7699-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fcce4857-4551-4b82-8cd1-e8d2438c7699" (UID: "fcce4857-4551-4b82-8cd1-e8d2438c7699"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:02:09.278415 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.278269    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcce4857-4551-4b82-8cd1-e8d2438c7699-kube-api-access-6qg92" (OuterVolumeSpecName: "kube-api-access-6qg92") pod "fcce4857-4551-4b82-8cd1-e8d2438c7699" (UID: "fcce4857-4551-4b82-8cd1-e8d2438c7699"). InnerVolumeSpecName "kube-api-access-6qg92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:02:09.376640 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.376611    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fcce4857-4551-4b82-8cd1-e8d2438c7699-metrics-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:09.376640 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.376635    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fcce4857-4551-4b82-8cd1-e8d2438c7699-openshift-service-ca\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:09.376640 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.376644    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fcce4857-4551-4b82-8cd1-e8d2438c7699-maas-api-tls\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:09.376866 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.376652    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qg92\" (UniqueName: \"kubernetes.io/projected/fcce4857-4551-4b82-8cd1-e8d2438c7699-kube-api-access-6qg92\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:09.769815 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.769776    2576 generic.go:356] "Generic (PLEG): container finished" podID="fcce4857-4551-4b82-8cd1-e8d2438c7699" containerID="278345ce84ea4ba22fc18e4d051a712290acada409198fc7607aed9d50ae5cce" exitCode=0
Sep 19 11:02:09.769987 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.769826    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"
Sep 19 11:02:09.769987 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.769865    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr" event={"ID":"fcce4857-4551-4b82-8cd1-e8d2438c7699","Type":"ContainerDied","Data":"278345ce84ea4ba22fc18e4d051a712290acada409198fc7607aed9d50ae5cce"}
Sep 19 11:02:09.769987 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.769923    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr" event={"ID":"fcce4857-4551-4b82-8cd1-e8d2438c7699","Type":"ContainerDied","Data":"abb724b5dd3ef1be3856f1fddb6a6b73fd76cf5318d2e876a7a70b31d8493632"}
Sep 19 11:02:09.769987 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.769951    2576 scope.go:117] "RemoveContainer" containerID="278345ce84ea4ba22fc18e4d051a712290acada409198fc7607aed9d50ae5cce"
Sep 19 11:02:09.782307 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.782287    2576 scope.go:117] "RemoveContainer" containerID="278345ce84ea4ba22fc18e4d051a712290acada409198fc7607aed9d50ae5cce"
Sep 19 11:02:09.782712 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:02:09.782691    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"278345ce84ea4ba22fc18e4d051a712290acada409198fc7607aed9d50ae5cce\": container with ID starting with 278345ce84ea4ba22fc18e4d051a712290acada409198fc7607aed9d50ae5cce not found: ID does not exist" containerID="278345ce84ea4ba22fc18e4d051a712290acada409198fc7607aed9d50ae5cce"
Sep 19 11:02:09.782768 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.782720    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"278345ce84ea4ba22fc18e4d051a712290acada409198fc7607aed9d50ae5cce"} err="failed to get container status \"278345ce84ea4ba22fc18e4d051a712290acada409198fc7607aed9d50ae5cce\": rpc error: code = NotFound desc = could not find container \"278345ce84ea4ba22fc18e4d051a712290acada409198fc7607aed9d50ae5cce\": container with ID starting with 278345ce84ea4ba22fc18e4d051a712290acada409198fc7607aed9d50ae5cce not found: ID does not exist"
Sep 19 11:02:09.792377 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.792356    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"]
Sep 19 11:02:09.796770 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:09.796750    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-20c8d9-6b76bdcf56-xjdxr"]
Sep 19 11:02:11.006644 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:11.006607    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcce4857-4551-4b82-8cd1-e8d2438c7699" path="/var/lib/kubelet/pods/fcce4857-4551-4b82-8cd1-e8d2438c7699/volumes"
Sep 19 11:02:15.541114 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:15.541082    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"]
Sep 19 11:02:15.541655 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:15.541301    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv" podUID="801a3275-0941-4eb7-a49c-4602ac446b96" containerName="istio-proxy" containerID="cri-o://e60cc43470331a049058f83c27ebdfc9443cb2bd0870d54cae290a3eb7a45330" gracePeriod=30
Sep 19 11:02:20.676583 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.676556    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 19 11:02:20.676890 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.676621    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv" podUID="801a3275-0941-4eb7-a49c-4602ac446b96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 19 11:02:20.802120 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.802098    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 11:02:20.840947 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.840916    2576 generic.go:356] "Generic (PLEG): container finished" podID="801a3275-0941-4eb7-a49c-4602ac446b96" containerID="e60cc43470331a049058f83c27ebdfc9443cb2bd0870d54cae290a3eb7a45330" exitCode=0
Sep 19 11:02:20.841130 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.840973    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"
Sep 19 11:02:20.841130 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.840986    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv" event={"ID":"801a3275-0941-4eb7-a49c-4602ac446b96","Type":"ContainerDied","Data":"e60cc43470331a049058f83c27ebdfc9443cb2bd0870d54cae290a3eb7a45330"}
Sep 19 11:02:20.841130 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.841081    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv" event={"ID":"801a3275-0941-4eb7-a49c-4602ac446b96","Type":"ContainerDied","Data":"9801c8058d6c9fc525d251ac22513a6e5d50643bfd9e73f6ba349753b5ab5498"}
Sep 19 11:02:20.841130 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.841106    2576 scope.go:117] "RemoveContainer" containerID="e60cc43470331a049058f83c27ebdfc9443cb2bd0870d54cae290a3eb7a45330"
Sep 19 11:02:20.852747 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.852714    2576 scope.go:117] "RemoveContainer" containerID="22645a86e938e2c66c85c00f49c8e8c4f934caa7ffaf7daf2fde0006f1da91a8"
Sep 19 11:02:20.862564 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.862550    2576 scope.go:117] "RemoveContainer" containerID="e60cc43470331a049058f83c27ebdfc9443cb2bd0870d54cae290a3eb7a45330"
Sep 19 11:02:20.862824 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:02:20.862806    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e60cc43470331a049058f83c27ebdfc9443cb2bd0870d54cae290a3eb7a45330\": container with ID starting with e60cc43470331a049058f83c27ebdfc9443cb2bd0870d54cae290a3eb7a45330 not found: ID does not exist" containerID="e60cc43470331a049058f83c27ebdfc9443cb2bd0870d54cae290a3eb7a45330"
Sep 19 11:02:20.862876 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.862835    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e60cc43470331a049058f83c27ebdfc9443cb2bd0870d54cae290a3eb7a45330"} err="failed to get container status \"e60cc43470331a049058f83c27ebdfc9443cb2bd0870d54cae290a3eb7a45330\": rpc error: code = NotFound desc = could not find container \"e60cc43470331a049058f83c27ebdfc9443cb2bd0870d54cae290a3eb7a45330\": container with ID starting with e60cc43470331a049058f83c27ebdfc9443cb2bd0870d54cae290a3eb7a45330 not found: ID does not exist"
Sep 19 11:02:20.862876 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.862859    2576 scope.go:117] "RemoveContainer" containerID="22645a86e938e2c66c85c00f49c8e8c4f934caa7ffaf7daf2fde0006f1da91a8"
Sep 19 11:02:20.863123 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:02:20.863104    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22645a86e938e2c66c85c00f49c8e8c4f934caa7ffaf7daf2fde0006f1da91a8\": container with ID starting with 22645a86e938e2c66c85c00f49c8e8c4f934caa7ffaf7daf2fde0006f1da91a8 not found: ID does not exist" containerID="22645a86e938e2c66c85c00f49c8e8c4f934caa7ffaf7daf2fde0006f1da91a8"
Sep 19 11:02:20.863193 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.863141    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22645a86e938e2c66c85c00f49c8e8c4f934caa7ffaf7daf2fde0006f1da91a8"} err="failed to get container status \"22645a86e938e2c66c85c00f49c8e8c4f934caa7ffaf7daf2fde0006f1da91a8\": rpc error: code = NotFound desc = could not find container \"22645a86e938e2c66c85c00f49c8e8c4f934caa7ffaf7daf2fde0006f1da91a8\": container with ID starting with 22645a86e938e2c66c85c00f49c8e8c4f934caa7ffaf7daf2fde0006f1da91a8 not found: ID does not exist"
Sep 19 11:02:20.896066 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.896028    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-istio-envoy\") pod \"801a3275-0941-4eb7-a49c-4602ac446b96\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") "
Sep 19 11:02:20.896161 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.896084    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/801a3275-0941-4eb7-a49c-4602ac446b96-istio-podinfo\") pod \"801a3275-0941-4eb7-a49c-4602ac446b96\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") "
Sep 19 11:02:20.896161 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.896104    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-workload-socket\") pod \"801a3275-0941-4eb7-a49c-4602ac446b96\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") "
Sep 19 11:02:20.896161 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.896120    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-credential-socket\") pod \"801a3275-0941-4eb7-a49c-4602ac446b96\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") "
Sep 19 11:02:20.896161 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.896158    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/801a3275-0941-4eb7-a49c-4602ac446b96-istiod-ca-cert\") pod \"801a3275-0941-4eb7-a49c-4602ac446b96\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") "
Sep 19 11:02:20.896376 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.896192    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-workload-certs\") pod \"801a3275-0941-4eb7-a49c-4602ac446b96\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") "
Sep 19 11:02:20.896376 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.896303    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-istio-data\") pod \"801a3275-0941-4eb7-a49c-4602ac446b96\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") "
Sep 19 11:02:20.896376 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.896331    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrwm7\" (UniqueName: \"kubernetes.io/projected/801a3275-0941-4eb7-a49c-4602ac446b96-kube-api-access-xrwm7\") pod \"801a3275-0941-4eb7-a49c-4602ac446b96\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") "
Sep 19 11:02:20.896376 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.896370    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/801a3275-0941-4eb7-a49c-4602ac446b96-istio-token\") pod \"801a3275-0941-4eb7-a49c-4602ac446b96\" (UID: \"801a3275-0941-4eb7-a49c-4602ac446b96\") "
Sep 19 11:02:20.896595 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.896396    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "801a3275-0941-4eb7-a49c-4602ac446b96" (UID: "801a3275-0941-4eb7-a49c-4602ac446b96"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:02:20.896595 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.896443    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "801a3275-0941-4eb7-a49c-4602ac446b96" (UID: "801a3275-0941-4eb7-a49c-4602ac446b96"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:02:20.896595 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.896539    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "801a3275-0941-4eb7-a49c-4602ac446b96" (UID: "801a3275-0941-4eb7-a49c-4602ac446b96"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:02:20.896595 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.896555    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/801a3275-0941-4eb7-a49c-4602ac446b96-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "801a3275-0941-4eb7-a49c-4602ac446b96" (UID: "801a3275-0941-4eb7-a49c-4602ac446b96"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:02:20.897014 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.896952    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:20.897014 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.896974    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:20.897014 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.896990    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/801a3275-0941-4eb7-a49c-4602ac446b96-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:20.897014 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.897004    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:20.897314 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.897290    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-istio-data" (OuterVolumeSpecName: "istio-data") pod "801a3275-0941-4eb7-a49c-4602ac446b96" (UID: "801a3275-0941-4eb7-a49c-4602ac446b96"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:02:20.898409 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.898381    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/801a3275-0941-4eb7-a49c-4602ac446b96-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "801a3275-0941-4eb7-a49c-4602ac446b96" (UID: "801a3275-0941-4eb7-a49c-4602ac446b96"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 11:02:20.898494 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.898435    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "801a3275-0941-4eb7-a49c-4602ac446b96" (UID: "801a3275-0941-4eb7-a49c-4602ac446b96"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:02:20.898540 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.898499    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/801a3275-0941-4eb7-a49c-4602ac446b96-kube-api-access-xrwm7" (OuterVolumeSpecName: "kube-api-access-xrwm7") pod "801a3275-0941-4eb7-a49c-4602ac446b96" (UID: "801a3275-0941-4eb7-a49c-4602ac446b96"). InnerVolumeSpecName "kube-api-access-xrwm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:02:20.898577 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.898558    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/801a3275-0941-4eb7-a49c-4602ac446b96-istio-token" (OuterVolumeSpecName: "istio-token") pod "801a3275-0941-4eb7-a49c-4602ac446b96" (UID: "801a3275-0941-4eb7-a49c-4602ac446b96"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:02:20.998136 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.998107    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/801a3275-0941-4eb7-a49c-4602ac446b96-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:20.998136 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.998131    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:20.998136 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.998140    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/801a3275-0941-4eb7-a49c-4602ac446b96-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:20.998136 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.998148    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/801a3275-0941-4eb7-a49c-4602ac446b96-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:20.998406 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:20.998156    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrwm7\" (UniqueName: \"kubernetes.io/projected/801a3275-0941-4eb7-a49c-4602ac446b96-kube-api-access-xrwm7\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:21.157097 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:21.157006    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"]
Sep 19 11:02:21.162678 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:21.162651    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-20c8d9-openshift-default-75b7c6ffbb-hjsnv"]
Sep 19 11:02:23.002061 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:23.002021    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="801a3275-0941-4eb7-a49c-4602ac446b96" path="/var/lib/kubelet/pods/801a3275-0941-4eb7-a49c-4602ac446b96/volumes"
Sep 19 11:02:26.297556 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.297504    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"]
Sep 19 11:02:26.298029 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.297827    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg" podUID="4849b40d-7bd8-41e8-a6dd-19f6c749904d" containerName="maas-api" containerID="cri-o://3f34870d0eacaef04906e48081f33af7c1887f50e760b5f4d6c96762fa571248" gracePeriod=30
Sep 19 11:02:26.565097 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.565075    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"
Sep 19 11:02:26.650640 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.650602    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4849b40d-7bd8-41e8-a6dd-19f6c749904d-openshift-service-ca\") pod \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\" (UID: \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\") "
Sep 19 11:02:26.650844 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.650698    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmqqc\" (UniqueName: \"kubernetes.io/projected/4849b40d-7bd8-41e8-a6dd-19f6c749904d-kube-api-access-zmqqc\") pod \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\" (UID: \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\") "
Sep 19 11:02:26.650844 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.650794    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4849b40d-7bd8-41e8-a6dd-19f6c749904d-metrics-certs\") pod \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\" (UID: \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\") "
Sep 19 11:02:26.650844 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.650841    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4849b40d-7bd8-41e8-a6dd-19f6c749904d-maas-api-tls\") pod \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\" (UID: \"4849b40d-7bd8-41e8-a6dd-19f6c749904d\") "
Sep 19 11:02:26.651031 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.650990    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4849b40d-7bd8-41e8-a6dd-19f6c749904d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4849b40d-7bd8-41e8-a6dd-19f6c749904d" (UID: "4849b40d-7bd8-41e8-a6dd-19f6c749904d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:02:26.651315 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.651290    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4849b40d-7bd8-41e8-a6dd-19f6c749904d-openshift-service-ca\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:26.653795 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.653760    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4849b40d-7bd8-41e8-a6dd-19f6c749904d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4849b40d-7bd8-41e8-a6dd-19f6c749904d" (UID: "4849b40d-7bd8-41e8-a6dd-19f6c749904d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:02:26.653928 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.653897    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4849b40d-7bd8-41e8-a6dd-19f6c749904d-kube-api-access-zmqqc" (OuterVolumeSpecName: "kube-api-access-zmqqc") pod "4849b40d-7bd8-41e8-a6dd-19f6c749904d" (UID: "4849b40d-7bd8-41e8-a6dd-19f6c749904d"). InnerVolumeSpecName "kube-api-access-zmqqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:02:26.653928 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.653904    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4849b40d-7bd8-41e8-a6dd-19f6c749904d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4849b40d-7bd8-41e8-a6dd-19f6c749904d" (UID: "4849b40d-7bd8-41e8-a6dd-19f6c749904d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:02:26.752490 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.752458    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmqqc\" (UniqueName: \"kubernetes.io/projected/4849b40d-7bd8-41e8-a6dd-19f6c749904d-kube-api-access-zmqqc\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:26.752490 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.752487    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4849b40d-7bd8-41e8-a6dd-19f6c749904d-metrics-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:26.752667 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.752497    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4849b40d-7bd8-41e8-a6dd-19f6c749904d-maas-api-tls\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:26.880844 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.880808    2576 generic.go:356] "Generic (PLEG): container finished" podID="4849b40d-7bd8-41e8-a6dd-19f6c749904d" containerID="3f34870d0eacaef04906e48081f33af7c1887f50e760b5f4d6c96762fa571248" exitCode=0
Sep 19 11:02:26.881026 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.880851    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"
Sep 19 11:02:26.881026 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.880881    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg" event={"ID":"4849b40d-7bd8-41e8-a6dd-19f6c749904d","Type":"ContainerDied","Data":"3f34870d0eacaef04906e48081f33af7c1887f50e760b5f4d6c96762fa571248"}
Sep 19 11:02:26.881026 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.880934    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg" event={"ID":"4849b40d-7bd8-41e8-a6dd-19f6c749904d","Type":"ContainerDied","Data":"0b0f921421efff32c01b8c849284a34fa8e743b90d3feb379bcbef8431a6021a"}
Sep 19 11:02:26.881026 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.880956    2576 scope.go:117] "RemoveContainer" containerID="3f34870d0eacaef04906e48081f33af7c1887f50e760b5f4d6c96762fa571248"
Sep 19 11:02:26.892791 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.892770    2576 scope.go:117] "RemoveContainer" containerID="3f34870d0eacaef04906e48081f33af7c1887f50e760b5f4d6c96762fa571248"
Sep 19 11:02:26.893112 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:02:26.893089    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f34870d0eacaef04906e48081f33af7c1887f50e760b5f4d6c96762fa571248\": container with ID starting with 3f34870d0eacaef04906e48081f33af7c1887f50e760b5f4d6c96762fa571248 not found: ID does not exist" containerID="3f34870d0eacaef04906e48081f33af7c1887f50e760b5f4d6c96762fa571248"
Sep 19 11:02:26.893190 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.893123    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f34870d0eacaef04906e48081f33af7c1887f50e760b5f4d6c96762fa571248"} err="failed to get container status \"3f34870d0eacaef04906e48081f33af7c1887f50e760b5f4d6c96762fa571248\": rpc error: code = NotFound desc = could not find container \"3f34870d0eacaef04906e48081f33af7c1887f50e760b5f4d6c96762fa571248\": container with ID starting with 3f34870d0eacaef04906e48081f33af7c1887f50e760b5f4d6c96762fa571248 not found: ID does not exist"
Sep 19 11:02:26.908251 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.908224    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"]
Sep 19 11:02:26.912726 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:26.912704    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8ddce5-65fb8c9ccb-pfmdg"]
Sep 19 11:02:27.006134 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:27.006096    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4849b40d-7bd8-41e8-a6dd-19f6c749904d" path="/var/lib/kubelet/pods/4849b40d-7bd8-41e8-a6dd-19f6c749904d/volumes"
Sep 19 11:02:55.068058 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:55.068013    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck"]
Sep 19 11:02:55.068626 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:55.068244    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck" podUID="61fc1267-dfbc-480c-b6df-fdc193061041" containerName="payload-processing" containerID="cri-o://42f892eeb8906de49d80ac4ecf44e5ced8d147432b4691cd6b36d6fbb141a7d5" gracePeriod=30
Sep 19 11:02:55.075496 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:55.075463    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx"]
Sep 19 11:02:55.075740 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:55.075693    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx" podUID="55dca516-aa13-4415-b070-c3e64fc30de9" containerName="payload-pre-processing" containerID="cri-o://51e03c08e6288f9164ab8f4f9cfe0fe6f78d2fba4359738ba1a4b0e80f7748ff" gracePeriod=30
Sep 19 11:02:55.413851 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:55.413821    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx"
Sep 19 11:02:55.416781 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:55.416763    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck"
Sep 19 11:02:55.526203 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:55.526173    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61fc1267-dfbc-480c-b6df-fdc193061041-plugins-config-volume\") pod \"61fc1267-dfbc-480c-b6df-fdc193061041\" (UID: \"61fc1267-dfbc-480c-b6df-fdc193061041\") "
Sep 19 11:02:55.526361 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:55.526254    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55dca516-aa13-4415-b070-c3e64fc30de9-plugins-config-volume\") pod \"55dca516-aa13-4415-b070-c3e64fc30de9\" (UID: \"55dca516-aa13-4415-b070-c3e64fc30de9\") "
Sep 19 11:02:55.526426 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:55.526407    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2p955\" (UniqueName: \"kubernetes.io/projected/55dca516-aa13-4415-b070-c3e64fc30de9-kube-api-access-2p955\") pod \"55dca516-aa13-4415-b070-c3e64fc30de9\" (UID: \"55dca516-aa13-4415-b070-c3e64fc30de9\") "
Sep 19 11:02:55.526517 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:55.526502    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rskzn\" (UniqueName: \"kubernetes.io/projected/61fc1267-dfbc-480c-b6df-fdc193061041-kube-api-access-rskzn\") pod \"61fc1267-dfbc-480c-b6df-fdc193061041\" (UID: \"61fc1267-dfbc-480c-b6df-fdc193061041\") "
Sep 19 11:02:55.526571 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:55.526550    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55dca516-aa13-4415-b070-c3e64fc30de9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "55dca516-aa13-4415-b070-c3e64fc30de9" (UID: "55dca516-aa13-4415-b070-c3e64fc30de9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:02:55.526571 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:55.526550    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61fc1267-dfbc-480c-b6df-fdc193061041-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "61fc1267-dfbc-480c-b6df-fdc193061041" (UID: "61fc1267-dfbc-480c-b6df-fdc193061041"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:02:55.526886 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:55.526860    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61fc1267-dfbc-480c-b6df-fdc193061041-plugins-config-volume\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:55.526886 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:55.526885    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55dca516-aa13-4415-b070-c3e64fc30de9-plugins-config-volume\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:55.528569 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:55.528545    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55dca516-aa13-4415-b070-c3e64fc30de9-kube-api-access-2p955" (OuterVolumeSpecName: "kube-api-access-2p955") pod "55dca516-aa13-4415-b070-c3e64fc30de9" (UID: "55dca516-aa13-4415-b070-c3e64fc30de9"). InnerVolumeSpecName "kube-api-access-2p955". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:02:55.528650 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:55.528605    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61fc1267-dfbc-480c-b6df-fdc193061041-kube-api-access-rskzn" (OuterVolumeSpecName: "kube-api-access-rskzn") pod "61fc1267-dfbc-480c-b6df-fdc193061041" (UID: "61fc1267-dfbc-480c-b6df-fdc193061041"). InnerVolumeSpecName "kube-api-access-rskzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:02:55.627974 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:55.627948    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2p955\" (UniqueName: \"kubernetes.io/projected/55dca516-aa13-4415-b070-c3e64fc30de9-kube-api-access-2p955\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:55.627974 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:55.627972    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rskzn\" (UniqueName: \"kubernetes.io/projected/61fc1267-dfbc-480c-b6df-fdc193061041-kube-api-access-rskzn\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:02:56.043912 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:56.043875    2576 generic.go:356] "Generic (PLEG): container finished" podID="61fc1267-dfbc-480c-b6df-fdc193061041" containerID="42f892eeb8906de49d80ac4ecf44e5ced8d147432b4691cd6b36d6fbb141a7d5" exitCode=0
Sep 19 11:02:56.044129 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:56.043933    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck"
Sep 19 11:02:56.044129 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:56.043969    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck" event={"ID":"61fc1267-dfbc-480c-b6df-fdc193061041","Type":"ContainerDied","Data":"42f892eeb8906de49d80ac4ecf44e5ced8d147432b4691cd6b36d6fbb141a7d5"}
Sep 19 11:02:56.044129 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:56.044026    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck" event={"ID":"61fc1267-dfbc-480c-b6df-fdc193061041","Type":"ContainerDied","Data":"8b26cb79381f74037c27ebd7f43fa15256848cb0e4154224f28c5359eef942d8"}
Sep 19 11:02:56.044129 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:56.044076    2576 scope.go:117] "RemoveContainer" containerID="42f892eeb8906de49d80ac4ecf44e5ced8d147432b4691cd6b36d6fbb141a7d5"
Sep 19 11:02:56.045610 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:56.045578    2576 generic.go:356] "Generic (PLEG): container finished" podID="55dca516-aa13-4415-b070-c3e64fc30de9" containerID="51e03c08e6288f9164ab8f4f9cfe0fe6f78d2fba4359738ba1a4b0e80f7748ff" exitCode=0
Sep 19 11:02:56.045610 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:56.045604    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx"
Sep 19 11:02:56.045965 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:56.045661    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx" event={"ID":"55dca516-aa13-4415-b070-c3e64fc30de9","Type":"ContainerDied","Data":"51e03c08e6288f9164ab8f4f9cfe0fe6f78d2fba4359738ba1a4b0e80f7748ff"}
Sep 19 11:02:56.045965 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:56.045716    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx" event={"ID":"55dca516-aa13-4415-b070-c3e64fc30de9","Type":"ContainerDied","Data":"e7bbf396242ca0a98cf157f0898f200aa2f27de9e4fcea3d6d88b27221c0b4d4"}
Sep 19 11:02:56.055745 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:56.055715    2576 scope.go:117] "RemoveContainer" containerID="42f892eeb8906de49d80ac4ecf44e5ced8d147432b4691cd6b36d6fbb141a7d5"
Sep 19 11:02:56.056300 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:02:56.056032    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42f892eeb8906de49d80ac4ecf44e5ced8d147432b4691cd6b36d6fbb141a7d5\": container with ID starting with 42f892eeb8906de49d80ac4ecf44e5ced8d147432b4691cd6b36d6fbb141a7d5 not found: ID does not exist" containerID="42f892eeb8906de49d80ac4ecf44e5ced8d147432b4691cd6b36d6fbb141a7d5"
Sep 19 11:02:56.056300 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:56.056148    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42f892eeb8906de49d80ac4ecf44e5ced8d147432b4691cd6b36d6fbb141a7d5"} err="failed to get container status \"42f892eeb8906de49d80ac4ecf44e5ced8d147432b4691cd6b36d6fbb141a7d5\": rpc error: code = NotFound desc = could not find container \"42f892eeb8906de49d80ac4ecf44e5ced8d147432b4691cd6b36d6fbb141a7d5\": container with ID starting with 42f892eeb8906de49d80ac4ecf44e5ced8d147432b4691cd6b36d6fbb141a7d5 not found: ID does not exist"
Sep 19 11:02:56.056599 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:56.056176    2576 scope.go:117] "RemoveContainer" containerID="51e03c08e6288f9164ab8f4f9cfe0fe6f78d2fba4359738ba1a4b0e80f7748ff"
Sep 19 11:02:56.065226 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:56.065209    2576 scope.go:117] "RemoveContainer" containerID="51e03c08e6288f9164ab8f4f9cfe0fe6f78d2fba4359738ba1a4b0e80f7748ff"
Sep 19 11:02:56.065498 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:02:56.065481    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51e03c08e6288f9164ab8f4f9cfe0fe6f78d2fba4359738ba1a4b0e80f7748ff\": container with ID starting with 51e03c08e6288f9164ab8f4f9cfe0fe6f78d2fba4359738ba1a4b0e80f7748ff not found: ID does not exist" containerID="51e03c08e6288f9164ab8f4f9cfe0fe6f78d2fba4359738ba1a4b0e80f7748ff"
Sep 19 11:02:56.065577 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:56.065501    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51e03c08e6288f9164ab8f4f9cfe0fe6f78d2fba4359738ba1a4b0e80f7748ff"} err="failed to get container status \"51e03c08e6288f9164ab8f4f9cfe0fe6f78d2fba4359738ba1a4b0e80f7748ff\": rpc error: code = NotFound desc = could not find container \"51e03c08e6288f9164ab8f4f9cfe0fe6f78d2fba4359738ba1a4b0e80f7748ff\": container with ID starting with 51e03c08e6288f9164ab8f4f9cfe0fe6f78d2fba4359738ba1a4b0e80f7748ff not found: ID does not exist"
Sep 19 11:02:56.068895 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:56.068874    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck"]
Sep 19 11:02:56.073817 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:56.073797    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-1907c1-7fbfbf69c7-krzck"]
Sep 19 11:02:56.076784 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:02:56.076758    2576 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61fc1267_dfbc_480c_b6df_fdc193061041.slice/crio-8b26cb79381f74037c27ebd7f43fa15256848cb0e4154224f28c5359eef942d8\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55dca516_aa13_4415_b070_c3e64fc30de9.slice/crio-e7bbf396242ca0a98cf157f0898f200aa2f27de9e4fcea3d6d88b27221c0b4d4\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61fc1267_dfbc_480c_b6df_fdc193061041.slice\": RecentStats: unable to find data in memory cache]"
Sep 19 11:02:56.082131 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:56.082110    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx"]
Sep 19 11:02:56.085133 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:56.085113    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-1907c1-5f44c8d4d6-vm6jx"]
Sep 19 11:02:57.002108 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:57.002073    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55dca516-aa13-4415-b070-c3e64fc30de9" path="/var/lib/kubelet/pods/55dca516-aa13-4415-b070-c3e64fc30de9/volumes"
Sep 19 11:02:57.002442 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:02:57.002429    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61fc1267-dfbc-480c-b6df-fdc193061041" path="/var/lib/kubelet/pods/61fc1267-dfbc-480c-b6df-fdc193061041/volumes"
Sep 19 11:03:01.401771 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:01.401739    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"]
Sep 19 11:03:01.402273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:01.402000    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx" podUID="b368f1b5-0779-4f8f-86ff-d6016e2bdc84" containerName="istio-proxy" containerID="cri-o://fb56bb0a8579dba8d3fa78db3c86fa3c8a336f5edf5911b401d7327183d0b3b9" gracePeriod=30
Sep 19 11:03:04.683618 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:04.683583    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"]
Sep 19 11:03:04.684131 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:04.683932    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx" podUID="615a8f92-a0a9-4b40-b723-5422c035dda5" containerName="maas-api" containerID="cri-o://3d4d13ad37e495e420d436e0cae221fb6ffb9e23031e90a399e6c77084d4c9f6" gracePeriod=30
Sep 19 11:03:04.747311 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:04.747266    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw"]
Sep 19 11:03:04.749162 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:04.747522    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw" podUID="84c64e61-3aca-46d8-bfd6-065287bf0fb8" containerName="payload-pre-processing" containerID="cri-o://e4f821a4a56b069339bf40001b2df7178a6d97b882c4c3d710bfa7cf8c93b8f6" gracePeriod=30
Sep 19 11:03:05.029957 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.029936    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw"
Sep 19 11:03:05.105869 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.105839    2576 generic.go:356] "Generic (PLEG): container finished" podID="615a8f92-a0a9-4b40-b723-5422c035dda5" containerID="3d4d13ad37e495e420d436e0cae221fb6ffb9e23031e90a399e6c77084d4c9f6" exitCode=0
Sep 19 11:03:05.106085 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.105916    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx" event={"ID":"615a8f92-a0a9-4b40-b723-5422c035dda5","Type":"ContainerDied","Data":"3d4d13ad37e495e420d436e0cae221fb6ffb9e23031e90a399e6c77084d4c9f6"}
Sep 19 11:03:05.107584 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.107548    2576 generic.go:356] "Generic (PLEG): container finished" podID="84c64e61-3aca-46d8-bfd6-065287bf0fb8" containerID="e4f821a4a56b069339bf40001b2df7178a6d97b882c4c3d710bfa7cf8c93b8f6" exitCode=0
Sep 19 11:03:05.107739 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.107620    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw" event={"ID":"84c64e61-3aca-46d8-bfd6-065287bf0fb8","Type":"ContainerDied","Data":"e4f821a4a56b069339bf40001b2df7178a6d97b882c4c3d710bfa7cf8c93b8f6"}
Sep 19 11:03:05.107739 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.107659    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw" event={"ID":"84c64e61-3aca-46d8-bfd6-065287bf0fb8","Type":"ContainerDied","Data":"3f9c057453eeeb7e63b0a241808f967c7d795e7e0c2955a628525d6598f430f4"}
Sep 19 11:03:05.107739 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.107684    2576 scope.go:117] "RemoveContainer" containerID="e4f821a4a56b069339bf40001b2df7178a6d97b882c4c3d710bfa7cf8c93b8f6"
Sep 19 11:03:05.107922 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.107831    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw"
Sep 19 11:03:05.116480 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.116443    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2d2q\" (UniqueName: \"kubernetes.io/projected/84c64e61-3aca-46d8-bfd6-065287bf0fb8-kube-api-access-m2d2q\") pod \"84c64e61-3aca-46d8-bfd6-065287bf0fb8\" (UID: \"84c64e61-3aca-46d8-bfd6-065287bf0fb8\") "
Sep 19 11:03:05.116615 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.116524    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84c64e61-3aca-46d8-bfd6-065287bf0fb8-plugins-config-volume\") pod \"84c64e61-3aca-46d8-bfd6-065287bf0fb8\" (UID: \"84c64e61-3aca-46d8-bfd6-065287bf0fb8\") "
Sep 19 11:03:05.117177 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.117147    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84c64e61-3aca-46d8-bfd6-065287bf0fb8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "84c64e61-3aca-46d8-bfd6-065287bf0fb8" (UID: "84c64e61-3aca-46d8-bfd6-065287bf0fb8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:03:05.119508 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.119472    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84c64e61-3aca-46d8-bfd6-065287bf0fb8-kube-api-access-m2d2q" (OuterVolumeSpecName: "kube-api-access-m2d2q") pod "84c64e61-3aca-46d8-bfd6-065287bf0fb8" (UID: "84c64e61-3aca-46d8-bfd6-065287bf0fb8"). InnerVolumeSpecName "kube-api-access-m2d2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:03:05.120228 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.120210    2576 scope.go:117] "RemoveContainer" containerID="e4f821a4a56b069339bf40001b2df7178a6d97b882c4c3d710bfa7cf8c93b8f6"
Sep 19 11:03:05.120540 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:03:05.120517    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4f821a4a56b069339bf40001b2df7178a6d97b882c4c3d710bfa7cf8c93b8f6\": container with ID starting with e4f821a4a56b069339bf40001b2df7178a6d97b882c4c3d710bfa7cf8c93b8f6 not found: ID does not exist" containerID="e4f821a4a56b069339bf40001b2df7178a6d97b882c4c3d710bfa7cf8c93b8f6"
Sep 19 11:03:05.120649 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.120545    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4f821a4a56b069339bf40001b2df7178a6d97b882c4c3d710bfa7cf8c93b8f6"} err="failed to get container status \"e4f821a4a56b069339bf40001b2df7178a6d97b882c4c3d710bfa7cf8c93b8f6\": rpc error: code = NotFound desc = could not find container \"e4f821a4a56b069339bf40001b2df7178a6d97b882c4c3d710bfa7cf8c93b8f6\": container with ID starting with e4f821a4a56b069339bf40001b2df7178a6d97b882c4c3d710bfa7cf8c93b8f6 not found: ID does not exist"
Sep 19 11:03:05.166593 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.166570    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"
Sep 19 11:03:05.217556 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.217487    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tbxs\" (UniqueName: \"kubernetes.io/projected/615a8f92-a0a9-4b40-b723-5422c035dda5-kube-api-access-5tbxs\") pod \"615a8f92-a0a9-4b40-b723-5422c035dda5\" (UID: \"615a8f92-a0a9-4b40-b723-5422c035dda5\") "
Sep 19 11:03:05.217556 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.217545    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/615a8f92-a0a9-4b40-b723-5422c035dda5-maas-api-tls\") pod \"615a8f92-a0a9-4b40-b723-5422c035dda5\" (UID: \"615a8f92-a0a9-4b40-b723-5422c035dda5\") "
Sep 19 11:03:05.217698 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.217639    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/615a8f92-a0a9-4b40-b723-5422c035dda5-metrics-certs\") pod \"615a8f92-a0a9-4b40-b723-5422c035dda5\" (UID: \"615a8f92-a0a9-4b40-b723-5422c035dda5\") "
Sep 19 11:03:05.217760 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.217744    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/615a8f92-a0a9-4b40-b723-5422c035dda5-openshift-service-ca\") pod \"615a8f92-a0a9-4b40-b723-5422c035dda5\" (UID: \"615a8f92-a0a9-4b40-b723-5422c035dda5\") "
Sep 19 11:03:05.218124 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.218101    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/615a8f92-a0a9-4b40-b723-5422c035dda5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "615a8f92-a0a9-4b40-b723-5422c035dda5" (UID: "615a8f92-a0a9-4b40-b723-5422c035dda5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:03:05.218230 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.218164    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2d2q\" (UniqueName: \"kubernetes.io/projected/84c64e61-3aca-46d8-bfd6-065287bf0fb8-kube-api-access-m2d2q\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:03:05.218230 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.218186    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84c64e61-3aca-46d8-bfd6-065287bf0fb8-plugins-config-volume\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:03:05.219748 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.219727    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/615a8f92-a0a9-4b40-b723-5422c035dda5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "615a8f92-a0a9-4b40-b723-5422c035dda5" (UID: "615a8f92-a0a9-4b40-b723-5422c035dda5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:03:05.220183 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.220161    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/615a8f92-a0a9-4b40-b723-5422c035dda5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "615a8f92-a0a9-4b40-b723-5422c035dda5" (UID: "615a8f92-a0a9-4b40-b723-5422c035dda5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:03:05.220183 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.220167    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/615a8f92-a0a9-4b40-b723-5422c035dda5-kube-api-access-5tbxs" (OuterVolumeSpecName: "kube-api-access-5tbxs") pod "615a8f92-a0a9-4b40-b723-5422c035dda5" (UID: "615a8f92-a0a9-4b40-b723-5422c035dda5"). InnerVolumeSpecName "kube-api-access-5tbxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:03:05.319518 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.319485    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/615a8f92-a0a9-4b40-b723-5422c035dda5-openshift-service-ca\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:03:05.319518 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.319514    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tbxs\" (UniqueName: \"kubernetes.io/projected/615a8f92-a0a9-4b40-b723-5422c035dda5-kube-api-access-5tbxs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:03:05.319699 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.319528    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/615a8f92-a0a9-4b40-b723-5422c035dda5-maas-api-tls\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:03:05.319699 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.319540    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/615a8f92-a0a9-4b40-b723-5422c035dda5-metrics-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:03:05.430589 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.430558    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw"]
Sep 19 11:03:05.432625 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:05.432606    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-5116ec-695f49474-ds6qw"]
Sep 19 11:03:06.114431 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.114406    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"
Sep 19 11:03:06.114431 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.114429    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx" event={"ID":"615a8f92-a0a9-4b40-b723-5422c035dda5","Type":"ContainerDied","Data":"ccd7b968c9206e69ab5a5d6be8cafa5d4f6501f428878c1c2837013ec73269ac"}
Sep 19 11:03:06.114903 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.114492    2576 scope.go:117] "RemoveContainer" containerID="3d4d13ad37e495e420d436e0cae221fb6ffb9e23031e90a399e6c77084d4c9f6"
Sep 19 11:03:06.136691 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.136659    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"]
Sep 19 11:03:06.140168 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.140147    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-5116ec-7bfc87dc9b-hzdkx"]
Sep 19 11:03:06.648019 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.647996    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 11:03:06.731654 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.731579    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-envoy\") pod \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") "
Sep 19 11:03:06.731654 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.731623    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wb64f\" (UniqueName: \"kubernetes.io/projected/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-kube-api-access-wb64f\") pod \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") "
Sep 19 11:03:06.731654 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.731646    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-token\") pod \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") "
Sep 19 11:03:06.731938 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.731668    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-workload-socket\") pod \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") "
Sep 19 11:03:06.731938 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.731688    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-credential-socket\") pod \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") "
Sep 19 11:03:06.731938 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.731705    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-data\") pod \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") "
Sep 19 11:03:06.731938 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.731743    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istiod-ca-cert\") pod \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") "
Sep 19 11:03:06.731938 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.731799    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-workload-certs\") pod \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") "
Sep 19 11:03:06.731938 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.731882    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-podinfo\") pod \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\" (UID: \"b368f1b5-0779-4f8f-86ff-d6016e2bdc84\") "
Sep 19 11:03:06.732298 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.731933    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b368f1b5-0779-4f8f-86ff-d6016e2bdc84" (UID: "b368f1b5-0779-4f8f-86ff-d6016e2bdc84"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:03:06.732298 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.732125    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-data" (OuterVolumeSpecName: "istio-data") pod "b368f1b5-0779-4f8f-86ff-d6016e2bdc84" (UID: "b368f1b5-0779-4f8f-86ff-d6016e2bdc84"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:03:06.732298 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.732239    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b368f1b5-0779-4f8f-86ff-d6016e2bdc84" (UID: "b368f1b5-0779-4f8f-86ff-d6016e2bdc84"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:03:06.732639 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.732334    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b368f1b5-0779-4f8f-86ff-d6016e2bdc84" (UID: "b368f1b5-0779-4f8f-86ff-d6016e2bdc84"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:03:06.732639 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.732469    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:03:06.732639 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.732491    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:03:06.732639 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.732497    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b368f1b5-0779-4f8f-86ff-d6016e2bdc84" (UID: "b368f1b5-0779-4f8f-86ff-d6016e2bdc84"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:03:06.732639 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.732506    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:03:06.732639 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.732542    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:03:06.733914 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.733885    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-kube-api-access-wb64f" (OuterVolumeSpecName: "kube-api-access-wb64f") pod "b368f1b5-0779-4f8f-86ff-d6016e2bdc84" (UID: "b368f1b5-0779-4f8f-86ff-d6016e2bdc84"). InnerVolumeSpecName "kube-api-access-wb64f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:03:06.734143 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.734116    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b368f1b5-0779-4f8f-86ff-d6016e2bdc84" (UID: "b368f1b5-0779-4f8f-86ff-d6016e2bdc84"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 11:03:06.734224 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.734186    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b368f1b5-0779-4f8f-86ff-d6016e2bdc84" (UID: "b368f1b5-0779-4f8f-86ff-d6016e2bdc84"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:03:06.734224 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.734200    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-token" (OuterVolumeSpecName: "istio-token") pod "b368f1b5-0779-4f8f-86ff-d6016e2bdc84" (UID: "b368f1b5-0779-4f8f-86ff-d6016e2bdc84"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:03:06.833268 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.833238    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wb64f\" (UniqueName: \"kubernetes.io/projected/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-kube-api-access-wb64f\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:03:06.833268 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.833265    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:03:06.833432 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.833278    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:03:06.833432 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.833290    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:03:06.833432 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:06.833301    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b368f1b5-0779-4f8f-86ff-d6016e2bdc84-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:03:07.002115 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:07.002028    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="615a8f92-a0a9-4b40-b723-5422c035dda5" path="/var/lib/kubelet/pods/615a8f92-a0a9-4b40-b723-5422c035dda5/volumes"
Sep 19 11:03:07.002395 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:07.002381    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84c64e61-3aca-46d8-bfd6-065287bf0fb8" path="/var/lib/kubelet/pods/84c64e61-3aca-46d8-bfd6-065287bf0fb8/volumes"
Sep 19 11:03:07.124932 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:07.124909    2576 generic.go:356] "Generic (PLEG): container finished" podID="b368f1b5-0779-4f8f-86ff-d6016e2bdc84" containerID="fb56bb0a8579dba8d3fa78db3c86fa3c8a336f5edf5911b401d7327183d0b3b9" exitCode=0
Sep 19 11:03:07.125395 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:07.124950    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"
Sep 19 11:03:07.125395 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:07.124945    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx" event={"ID":"b368f1b5-0779-4f8f-86ff-d6016e2bdc84","Type":"ContainerDied","Data":"fb56bb0a8579dba8d3fa78db3c86fa3c8a336f5edf5911b401d7327183d0b3b9"}
Sep 19 11:03:07.125395 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:07.125086    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx" event={"ID":"b368f1b5-0779-4f8f-86ff-d6016e2bdc84","Type":"ContainerDied","Data":"9ba09a78ebf8407066fbce62a4c2abc2cc51a103a5a8245f75eb17aa50695d19"}
Sep 19 11:03:07.125395 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:07.125108    2576 scope.go:117] "RemoveContainer" containerID="fb56bb0a8579dba8d3fa78db3c86fa3c8a336f5edf5911b401d7327183d0b3b9"
Sep 19 11:03:07.134784 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:07.134765    2576 scope.go:117] "RemoveContainer" containerID="af2254b9483c8ed0f892fa35eb348de6ede840efa8f7fc5e6190625d4efd7c18"
Sep 19 11:03:07.142727 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:07.142706    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"]
Sep 19 11:03:07.143951 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:07.143898    2576 scope.go:117] "RemoveContainer" containerID="fb56bb0a8579dba8d3fa78db3c86fa3c8a336f5edf5911b401d7327183d0b3b9"
Sep 19 11:03:07.144426 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:03:07.144258    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb56bb0a8579dba8d3fa78db3c86fa3c8a336f5edf5911b401d7327183d0b3b9\": container with ID starting with fb56bb0a8579dba8d3fa78db3c86fa3c8a336f5edf5911b401d7327183d0b3b9 not found: ID does not exist" containerID="fb56bb0a8579dba8d3fa78db3c86fa3c8a336f5edf5911b401d7327183d0b3b9"
Sep 19 11:03:07.144426 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:07.144291    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb56bb0a8579dba8d3fa78db3c86fa3c8a336f5edf5911b401d7327183d0b3b9"} err="failed to get container status \"fb56bb0a8579dba8d3fa78db3c86fa3c8a336f5edf5911b401d7327183d0b3b9\": rpc error: code = NotFound desc = could not find container \"fb56bb0a8579dba8d3fa78db3c86fa3c8a336f5edf5911b401d7327183d0b3b9\": container with ID starting with fb56bb0a8579dba8d3fa78db3c86fa3c8a336f5edf5911b401d7327183d0b3b9 not found: ID does not exist"
Sep 19 11:03:07.144426 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:07.144314    2576 scope.go:117] "RemoveContainer" containerID="af2254b9483c8ed0f892fa35eb348de6ede840efa8f7fc5e6190625d4efd7c18"
Sep 19 11:03:07.144628 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:03:07.144608    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af2254b9483c8ed0f892fa35eb348de6ede840efa8f7fc5e6190625d4efd7c18\": container with ID starting with af2254b9483c8ed0f892fa35eb348de6ede840efa8f7fc5e6190625d4efd7c18 not found: ID does not exist" containerID="af2254b9483c8ed0f892fa35eb348de6ede840efa8f7fc5e6190625d4efd7c18"
Sep 19 11:03:07.144689 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:07.144629    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af2254b9483c8ed0f892fa35eb348de6ede840efa8f7fc5e6190625d4efd7c18"} err="failed to get container status \"af2254b9483c8ed0f892fa35eb348de6ede840efa8f7fc5e6190625d4efd7c18\": rpc error: code = NotFound desc = could not find container \"af2254b9483c8ed0f892fa35eb348de6ede840efa8f7fc5e6190625d4efd7c18\": container with ID starting with af2254b9483c8ed0f892fa35eb348de6ede840efa8f7fc5e6190625d4efd7c18 not found: ID does not exist"
Sep 19 11:03:07.146055 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:07.146014    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-1907c1-openshift-default-5bbd9698bb-xg5zx"]
Sep 19 11:03:09.002892 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:09.002843    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b368f1b5-0779-4f8f-86ff-d6016e2bdc84" path="/var/lib/kubelet/pods/b368f1b5-0779-4f8f-86ff-d6016e2bdc84/volumes"
Sep 19 11:03:35.814393 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814362    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7971-09edab"]
Sep 19 11:03:35.814786 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814764    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e94ce1b3-43d9-4712-902d-a60ba1ff5e6b" containerName="maas-api"
Sep 19 11:03:35.814786 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814776    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e94ce1b3-43d9-4712-902d-a60ba1ff5e6b" containerName="maas-api"
Sep 19 11:03:35.814860 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814792    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b368f1b5-0779-4f8f-86ff-d6016e2bdc84" containerName="istio-proxy"
Sep 19 11:03:35.814860 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814797    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b368f1b5-0779-4f8f-86ff-d6016e2bdc84" containerName="istio-proxy"
Sep 19 11:03:35.814860 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814804    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b368f1b5-0779-4f8f-86ff-d6016e2bdc84" containerName="istio-proxy"
Sep 19 11:03:35.814860 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814809    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b368f1b5-0779-4f8f-86ff-d6016e2bdc84" containerName="istio-proxy"
Sep 19 11:03:35.814860 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814816    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55dca516-aa13-4415-b070-c3e64fc30de9" containerName="payload-pre-processing"
Sep 19 11:03:35.814860 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814821    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="55dca516-aa13-4415-b070-c3e64fc30de9" containerName="payload-pre-processing"
Sep 19 11:03:35.814860 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814827    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="801a3275-0941-4eb7-a49c-4602ac446b96" containerName="istio-proxy"
Sep 19 11:03:35.814860 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814831    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="801a3275-0941-4eb7-a49c-4602ac446b96" containerName="istio-proxy"
Sep 19 11:03:35.814860 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814837    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4849b40d-7bd8-41e8-a6dd-19f6c749904d" containerName="maas-api"
Sep 19 11:03:35.814860 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814842    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4849b40d-7bd8-41e8-a6dd-19f6c749904d" containerName="maas-api"
Sep 19 11:03:35.814860 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814849    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d763461-6001-4ef2-9588-1b76b66a3cf4" containerName="istio-proxy"
Sep 19 11:03:35.814860 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814854    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d763461-6001-4ef2-9588-1b76b66a3cf4" containerName="istio-proxy"
Sep 19 11:03:35.814860 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814862    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcce4857-4551-4b82-8cd1-e8d2438c7699" containerName="maas-api"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814868    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcce4857-4551-4b82-8cd1-e8d2438c7699" containerName="maas-api"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814878    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a51a259-d565-4254-9176-dc7205fc61ac" containerName="istio-proxy"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814882    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a51a259-d565-4254-9176-dc7205fc61ac" containerName="istio-proxy"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814892    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84c64e61-3aca-46d8-bfd6-065287bf0fb8" containerName="payload-pre-processing"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814897    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="84c64e61-3aca-46d8-bfd6-065287bf0fb8" containerName="payload-pre-processing"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814902    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="615a8f92-a0a9-4b40-b723-5422c035dda5" containerName="maas-api"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814907    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="615a8f92-a0a9-4b40-b723-5422c035dda5" containerName="maas-api"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814913    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61fc1267-dfbc-480c-b6df-fdc193061041" containerName="payload-processing"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814919    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="61fc1267-dfbc-480c-b6df-fdc193061041" containerName="payload-processing"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814925    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="801a3275-0941-4eb7-a49c-4602ac446b96" containerName="istio-proxy"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814929    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="801a3275-0941-4eb7-a49c-4602ac446b96" containerName="istio-proxy"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814935    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a51a259-d565-4254-9176-dc7205fc61ac" containerName="istio-proxy"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.814940    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a51a259-d565-4254-9176-dc7205fc61ac" containerName="istio-proxy"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.815014    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcce4857-4551-4b82-8cd1-e8d2438c7699" containerName="maas-api"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.815022    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="801a3275-0941-4eb7-a49c-4602ac446b96" containerName="istio-proxy"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.815027    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a51a259-d565-4254-9176-dc7205fc61ac" containerName="istio-proxy"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.815033    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e94ce1b3-43d9-4712-902d-a60ba1ff5e6b" containerName="maas-api"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.815057    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b368f1b5-0779-4f8f-86ff-d6016e2bdc84" containerName="istio-proxy"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.815062    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="801a3275-0941-4eb7-a49c-4602ac446b96" containerName="istio-proxy"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.815069    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4849b40d-7bd8-41e8-a6dd-19f6c749904d" containerName="maas-api"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.815074    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="55dca516-aa13-4415-b070-c3e64fc30de9" containerName="payload-pre-processing"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.815079    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="61fc1267-dfbc-480c-b6df-fdc193061041" containerName="payload-processing"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.815085    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="615a8f92-a0a9-4b40-b723-5422c035dda5" containerName="maas-api"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.815090    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d763461-6001-4ef2-9588-1b76b66a3cf4" containerName="istio-proxy"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.815097    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="84c64e61-3aca-46d8-bfd6-065287bf0fb8" containerName="payload-pre-processing"
Sep 19 11:03:35.815273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.815102    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d763461-6001-4ef2-9588-1b76b66a3cf4" containerName="istio-proxy"
Sep 19 11:03:35.819628 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.819611    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7971-09edab"
Sep 19 11:03:35.822023 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.821999    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-m27fx\""
Sep 19 11:03:35.823211 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.823187    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7971-09edab"]
Sep 19 11:03:35.883167 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.883136    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhzqx\" (UniqueName: \"kubernetes.io/projected/b966421c-0548-4a35-9834-ce833117df76-kube-api-access-mhzqx\") pod \"test-curl-7971-09edab\" (UID: \"b966421c-0548-4a35-9834-ce833117df76\") " pod="opendatahub/test-curl-7971-09edab"
Sep 19 11:03:35.983740 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.983706    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhzqx\" (UniqueName: \"kubernetes.io/projected/b966421c-0548-4a35-9834-ce833117df76-kube-api-access-mhzqx\") pod \"test-curl-7971-09edab\" (UID: \"b966421c-0548-4a35-9834-ce833117df76\") " pod="opendatahub/test-curl-7971-09edab"
Sep 19 11:03:35.991358 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:35.991327    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhzqx\" (UniqueName: \"kubernetes.io/projected/b966421c-0548-4a35-9834-ce833117df76-kube-api-access-mhzqx\") pod \"test-curl-7971-09edab\" (UID: \"b966421c-0548-4a35-9834-ce833117df76\") " pod="opendatahub/test-curl-7971-09edab"
Sep 19 11:03:36.130841 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:36.130814    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7971-09edab"
Sep 19 11:03:36.401381 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:36.401357    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7971-09edab"]
Sep 19 11:03:36.403438 ip-10-0-142-165 kubenswrapper[2576]: W0919 11:03:36.403402    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb966421c_0548_4a35_9834_ce833117df76.slice/crio-1b775d1685a20ae5f471d2d8c1507d4387c77e755d29441c4a6db19ce3d50424 WatchSource:0}: Error finding container 1b775d1685a20ae5f471d2d8c1507d4387c77e755d29441c4a6db19ce3d50424: Status 404 returned error can't find the container with id 1b775d1685a20ae5f471d2d8c1507d4387c77e755d29441c4a6db19ce3d50424
Sep 19 11:03:37.301229 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:37.301196    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7971-09edab" event={"ID":"b966421c-0548-4a35-9834-ce833117df76","Type":"ContainerStarted","Data":"c2bb0f9cd3e1868dce17bd6ad5381954729b632b73616f7e03efaf3c7fcf7312"}
Sep 19 11:03:37.301591 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:37.301240    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7971-09edab" event={"ID":"b966421c-0548-4a35-9834-ce833117df76","Type":"ContainerStarted","Data":"1b775d1685a20ae5f471d2d8c1507d4387c77e755d29441c4a6db19ce3d50424"}
Sep 19 11:03:37.314180 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:37.314133    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7971-09edab" podStartSLOduration=2.314113662 podStartE2EDuration="2.314113662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:03:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:03:37.313837958 +0000 UTC m=+1626.810144051" watchObservedRunningTime="2026-09-19 11:03:37.314113662 +0000 UTC m=+1626.810419754"
Sep 19 11:03:37.945339 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:37.945304    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7971-09edab"]
Sep 19 11:03:37.949428 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:37.949404    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7971-09edab"]
Sep 19 11:03:38.306732 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:03:38.306694    2576 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7971-09edab" container="test-curl-7971-09edab"
Sep 19 11:03:39.311348 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:39.311311    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7971-09edab" podUID="b966421c-0548-4a35-9834-ce833117df76" containerName="test-curl-7971-09edab" containerID="cri-o://c2bb0f9cd3e1868dce17bd6ad5381954729b632b73616f7e03efaf3c7fcf7312" gracePeriod=2
Sep 19 11:03:41.504590 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:41.504567    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7971-09edab"
Sep 19 11:03:41.531863 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:41.531840    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhzqx\" (UniqueName: \"kubernetes.io/projected/b966421c-0548-4a35-9834-ce833117df76-kube-api-access-mhzqx\") pod \"b966421c-0548-4a35-9834-ce833117df76\" (UID: \"b966421c-0548-4a35-9834-ce833117df76\") "
Sep 19 11:03:41.533848 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:41.533824    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b966421c-0548-4a35-9834-ce833117df76-kube-api-access-mhzqx" (OuterVolumeSpecName: "kube-api-access-mhzqx") pod "b966421c-0548-4a35-9834-ce833117df76" (UID: "b966421c-0548-4a35-9834-ce833117df76"). InnerVolumeSpecName "kube-api-access-mhzqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:03:41.633394 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:41.633372    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhzqx\" (UniqueName: \"kubernetes.io/projected/b966421c-0548-4a35-9834-ce833117df76-kube-api-access-mhzqx\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:03:42.328231 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:42.328198    2576 generic.go:356] "Generic (PLEG): container finished" podID="b966421c-0548-4a35-9834-ce833117df76" containerID="c2bb0f9cd3e1868dce17bd6ad5381954729b632b73616f7e03efaf3c7fcf7312" exitCode=137
Sep 19 11:03:42.328386 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:42.328248    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7971-09edab"
Sep 19 11:03:42.328386 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:42.328304    2576 scope.go:117] "RemoveContainer" containerID="c2bb0f9cd3e1868dce17bd6ad5381954729b632b73616f7e03efaf3c7fcf7312"
Sep 19 11:03:42.337838 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:42.337822    2576 scope.go:117] "RemoveContainer" containerID="c2bb0f9cd3e1868dce17bd6ad5381954729b632b73616f7e03efaf3c7fcf7312"
Sep 19 11:03:42.338141 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:03:42.338122    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2bb0f9cd3e1868dce17bd6ad5381954729b632b73616f7e03efaf3c7fcf7312\": container with ID starting with c2bb0f9cd3e1868dce17bd6ad5381954729b632b73616f7e03efaf3c7fcf7312 not found: ID does not exist" containerID="c2bb0f9cd3e1868dce17bd6ad5381954729b632b73616f7e03efaf3c7fcf7312"
Sep 19 11:03:42.338228 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:42.338148    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2bb0f9cd3e1868dce17bd6ad5381954729b632b73616f7e03efaf3c7fcf7312"} err="failed to get container status \"c2bb0f9cd3e1868dce17bd6ad5381954729b632b73616f7e03efaf3c7fcf7312\": rpc error: code = NotFound desc = could not find container \"c2bb0f9cd3e1868dce17bd6ad5381954729b632b73616f7e03efaf3c7fcf7312\": container with ID starting with c2bb0f9cd3e1868dce17bd6ad5381954729b632b73616f7e03efaf3c7fcf7312 not found: ID does not exist"
Sep 19 11:03:42.338534 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:03:42.338514    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7971-09edab\" is forbidden: User \"system:node:ip-10-0-142-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-165.ec2.internal' and this object" podUID="b966421c-0548-4a35-9834-ce833117df76" pod="opendatahub/test-curl-7971-09edab"
Sep 19 11:03:43.001160 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:03:43.001128    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b966421c-0548-4a35-9834-ce833117df76" path="/var/lib/kubelet/pods/b966421c-0548-4a35-9834-ce833117df76/volumes"
Sep 19 11:04:40.322906 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:40.322876    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7971-be2e10"]
Sep 19 11:04:40.323380 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:40.323362    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d763461-6001-4ef2-9588-1b76b66a3cf4" containerName="istio-proxy"
Sep 19 11:04:40.323426 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:40.323384    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d763461-6001-4ef2-9588-1b76b66a3cf4" containerName="istio-proxy"
Sep 19 11:04:40.323426 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:40.323405    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b966421c-0548-4a35-9834-ce833117df76" containerName="test-curl-7971-09edab"
Sep 19 11:04:40.323426 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:40.323413    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b966421c-0548-4a35-9834-ce833117df76" containerName="test-curl-7971-09edab"
Sep 19 11:04:40.323524 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:40.323517    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a51a259-d565-4254-9176-dc7205fc61ac" containerName="istio-proxy"
Sep 19 11:04:40.323558 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:40.323529    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b368f1b5-0779-4f8f-86ff-d6016e2bdc84" containerName="istio-proxy"
Sep 19 11:04:40.323558 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:40.323537    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b966421c-0548-4a35-9834-ce833117df76" containerName="test-curl-7971-09edab"
Sep 19 11:04:40.327760 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:40.327743    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7971-be2e10"
Sep 19 11:04:40.330188 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:40.330161    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-m27fx\""
Sep 19 11:04:40.332145 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:40.332122    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7971-be2e10"]
Sep 19 11:04:40.445857 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:40.445828    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nzvd\" (UniqueName: \"kubernetes.io/projected/004b17e0-a1d6-4edb-ae7c-95bd2c9a1855-kube-api-access-4nzvd\") pod \"test-curl-7971-be2e10\" (UID: \"004b17e0-a1d6-4edb-ae7c-95bd2c9a1855\") " pod="opendatahub/test-curl-7971-be2e10"
Sep 19 11:04:40.546743 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:40.546718    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nzvd\" (UniqueName: \"kubernetes.io/projected/004b17e0-a1d6-4edb-ae7c-95bd2c9a1855-kube-api-access-4nzvd\") pod \"test-curl-7971-be2e10\" (UID: \"004b17e0-a1d6-4edb-ae7c-95bd2c9a1855\") " pod="opendatahub/test-curl-7971-be2e10"
Sep 19 11:04:40.557957 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:40.557931    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nzvd\" (UniqueName: \"kubernetes.io/projected/004b17e0-a1d6-4edb-ae7c-95bd2c9a1855-kube-api-access-4nzvd\") pod \"test-curl-7971-be2e10\" (UID: \"004b17e0-a1d6-4edb-ae7c-95bd2c9a1855\") " pod="opendatahub/test-curl-7971-be2e10"
Sep 19 11:04:40.638909 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:40.638888    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7971-be2e10"
Sep 19 11:04:40.907028 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:40.906998    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7971-be2e10"]
Sep 19 11:04:40.908960 ip-10-0-142-165 kubenswrapper[2576]: W0919 11:04:40.908935    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod004b17e0_a1d6_4edb_ae7c_95bd2c9a1855.slice/crio-38bdcff4aa84ea9050ae039176fb3150348b9ff0303b1d1e54d48f4d2295fccf WatchSource:0}: Error finding container 38bdcff4aa84ea9050ae039176fb3150348b9ff0303b1d1e54d48f4d2295fccf: Status 404 returned error can't find the container with id 38bdcff4aa84ea9050ae039176fb3150348b9ff0303b1d1e54d48f4d2295fccf
Sep 19 11:04:41.640740 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:41.640706    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7971-be2e10" event={"ID":"004b17e0-a1d6-4edb-ae7c-95bd2c9a1855","Type":"ContainerStarted","Data":"310f6deb13200e412dcdb9345d155a1ce2665bf7bbb90efb9e7914c21aae959f"}
Sep 19 11:04:41.641229 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:41.640749    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7971-be2e10" event={"ID":"004b17e0-a1d6-4edb-ae7c-95bd2c9a1855","Type":"ContainerStarted","Data":"38bdcff4aa84ea9050ae039176fb3150348b9ff0303b1d1e54d48f4d2295fccf"}
Sep 19 11:04:41.655365 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:41.655311    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7971-be2e10" podStartSLOduration=1.6552967 podStartE2EDuration="1.6552967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:04:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:04:41.65331731 +0000 UTC m=+1691.149623401" watchObservedRunningTime="2026-09-19 11:04:41.6552967 +0000 UTC m=+1691.151602791"
Sep 19 11:04:42.219291 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:42.219257    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7971-be2e10"]
Sep 19 11:04:42.223490 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:42.223463    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7971-be2e10"]
Sep 19 11:04:42.337067 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:42.337030    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7971-e3721d"]
Sep 19 11:04:42.337484 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:42.337472    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="004b17e0-a1d6-4edb-ae7c-95bd2c9a1855" containerName="test-curl-7971-be2e10"
Sep 19 11:04:42.337531 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:42.337486    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="004b17e0-a1d6-4edb-ae7c-95bd2c9a1855" containerName="test-curl-7971-be2e10"
Sep 19 11:04:42.337567 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:42.337560    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="004b17e0-a1d6-4edb-ae7c-95bd2c9a1855" containerName="test-curl-7971-be2e10"
Sep 19 11:04:42.340693 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:42.340678    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7971-e3721d"
Sep 19 11:04:42.344432 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:42.344411    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7971-e3721d"]
Sep 19 11:04:42.462735 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:42.462703    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgnqf\" (UniqueName: \"kubernetes.io/projected/5e407739-e207-4224-954b-7fb927bb1e5d-kube-api-access-pgnqf\") pod \"test-curl-7971-e3721d\" (UID: \"5e407739-e207-4224-954b-7fb927bb1e5d\") " pod="opendatahub/test-curl-7971-e3721d"
Sep 19 11:04:42.563348 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:42.563319    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgnqf\" (UniqueName: \"kubernetes.io/projected/5e407739-e207-4224-954b-7fb927bb1e5d-kube-api-access-pgnqf\") pod \"test-curl-7971-e3721d\" (UID: \"5e407739-e207-4224-954b-7fb927bb1e5d\") " pod="opendatahub/test-curl-7971-e3721d"
Sep 19 11:04:42.570828 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:42.570805    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgnqf\" (UniqueName: \"kubernetes.io/projected/5e407739-e207-4224-954b-7fb927bb1e5d-kube-api-access-pgnqf\") pod \"test-curl-7971-e3721d\" (UID: \"5e407739-e207-4224-954b-7fb927bb1e5d\") " pod="opendatahub/test-curl-7971-e3721d"
Sep 19 11:04:42.645905 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:04:42.645853    2576 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7971-be2e10" container="test-curl-7971-be2e10"
Sep 19 11:04:42.651726 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:42.651708    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7971-e3721d"
Sep 19 11:04:42.922435 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:42.922411    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7971-e3721d"]
Sep 19 11:04:42.924524 ip-10-0-142-165 kubenswrapper[2576]: W0919 11:04:42.924492    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e407739_e207_4224_954b_7fb927bb1e5d.slice/crio-80d87c591de7c5e255fd76455fd41456fca4e23166c6cd102b134944475ae189 WatchSource:0}: Error finding container 80d87c591de7c5e255fd76455fd41456fca4e23166c6cd102b134944475ae189: Status 404 returned error can't find the container with id 80d87c591de7c5e255fd76455fd41456fca4e23166c6cd102b134944475ae189
Sep 19 11:04:43.652912 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:43.652876    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7971-e3721d" event={"ID":"5e407739-e207-4224-954b-7fb927bb1e5d","Type":"ContainerStarted","Data":"14e217fead784060a72ae9a703cfaf0388259201dae8e60696fbb4215699f530"}
Sep 19 11:04:43.653454 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:43.652925    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7971-e3721d" event={"ID":"5e407739-e207-4224-954b-7fb927bb1e5d","Type":"ContainerStarted","Data":"80d87c591de7c5e255fd76455fd41456fca4e23166c6cd102b134944475ae189"}
Sep 19 11:04:43.653454 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:43.653033    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7971-be2e10" podUID="004b17e0-a1d6-4edb-ae7c-95bd2c9a1855" containerName="test-curl-7971-be2e10" containerID="cri-o://310f6deb13200e412dcdb9345d155a1ce2665bf7bbb90efb9e7914c21aae959f" gracePeriod=2
Sep 19 11:04:43.665502 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:43.665460    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7971-e3721d" podStartSLOduration=1.665446084 podStartE2EDuration="1.665446084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:04:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:04:43.665022143 +0000 UTC m=+1693.161328234" watchObservedRunningTime="2026-09-19 11:04:43.665446084 +0000 UTC m=+1693.161752176"
Sep 19 11:04:44.232865 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:44.232786    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7971-e3721d"]
Sep 19 11:04:44.236708 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:44.236676    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7971-e3721d"]
Sep 19 11:04:44.657599 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:04:44.657569    2576 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7971-e3721d" container="test-curl-7971-e3721d"
Sep 19 11:04:45.664185 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:45.664150    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7971-e3721d" podUID="5e407739-e207-4224-954b-7fb927bb1e5d" containerName="test-curl-7971-e3721d" containerID="cri-o://14e217fead784060a72ae9a703cfaf0388259201dae8e60696fbb4215699f530" gracePeriod=2
Sep 19 11:04:45.845480 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:45.845459    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7971-be2e10"
Sep 19 11:04:45.890488 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:45.890466    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nzvd\" (UniqueName: \"kubernetes.io/projected/004b17e0-a1d6-4edb-ae7c-95bd2c9a1855-kube-api-access-4nzvd\") pod \"004b17e0-a1d6-4edb-ae7c-95bd2c9a1855\" (UID: \"004b17e0-a1d6-4edb-ae7c-95bd2c9a1855\") "
Sep 19 11:04:45.892588 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:45.892560    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/004b17e0-a1d6-4edb-ae7c-95bd2c9a1855-kube-api-access-4nzvd" (OuterVolumeSpecName: "kube-api-access-4nzvd") pod "004b17e0-a1d6-4edb-ae7c-95bd2c9a1855" (UID: "004b17e0-a1d6-4edb-ae7c-95bd2c9a1855"). InnerVolumeSpecName "kube-api-access-4nzvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:04:45.991552 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:45.991503    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nzvd\" (UniqueName: \"kubernetes.io/projected/004b17e0-a1d6-4edb-ae7c-95bd2c9a1855-kube-api-access-4nzvd\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:04:46.671348 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:46.671315    2576 generic.go:356] "Generic (PLEG): container finished" podID="004b17e0-a1d6-4edb-ae7c-95bd2c9a1855" containerID="310f6deb13200e412dcdb9345d155a1ce2665bf7bbb90efb9e7914c21aae959f" exitCode=137
Sep 19 11:04:46.671876 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:46.671368    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7971-be2e10"
Sep 19 11:04:46.671876 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:46.671422    2576 scope.go:117] "RemoveContainer" containerID="310f6deb13200e412dcdb9345d155a1ce2665bf7bbb90efb9e7914c21aae959f"
Sep 19 11:04:46.681740 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:46.681724    2576 scope.go:117] "RemoveContainer" containerID="310f6deb13200e412dcdb9345d155a1ce2665bf7bbb90efb9e7914c21aae959f"
Sep 19 11:04:46.682006 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:04:46.681988    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"310f6deb13200e412dcdb9345d155a1ce2665bf7bbb90efb9e7914c21aae959f\": container with ID starting with 310f6deb13200e412dcdb9345d155a1ce2665bf7bbb90efb9e7914c21aae959f not found: ID does not exist" containerID="310f6deb13200e412dcdb9345d155a1ce2665bf7bbb90efb9e7914c21aae959f"
Sep 19 11:04:46.682070 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:46.682013    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"310f6deb13200e412dcdb9345d155a1ce2665bf7bbb90efb9e7914c21aae959f"} err="failed to get container status \"310f6deb13200e412dcdb9345d155a1ce2665bf7bbb90efb9e7914c21aae959f\": rpc error: code = NotFound desc = could not find container \"310f6deb13200e412dcdb9345d155a1ce2665bf7bbb90efb9e7914c21aae959f\": container with ID starting with 310f6deb13200e412dcdb9345d155a1ce2665bf7bbb90efb9e7914c21aae959f not found: ID does not exist"
Sep 19 11:04:46.682465 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:04:46.682444    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7971-be2e10\" is forbidden: User \"system:node:ip-10-0-142-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-165.ec2.internal' and this object" podUID="004b17e0-a1d6-4edb-ae7c-95bd2c9a1855" pod="opendatahub/test-curl-7971-be2e10"
Sep 19 11:04:47.001084 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:47.000999    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="004b17e0-a1d6-4edb-ae7c-95bd2c9a1855" path="/var/lib/kubelet/pods/004b17e0-a1d6-4edb-ae7c-95bd2c9a1855/volumes"
Sep 19 11:04:47.839056 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:47.839018    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7971-e3721d"
Sep 19 11:04:47.907832 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:47.907805    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgnqf\" (UniqueName: \"kubernetes.io/projected/5e407739-e207-4224-954b-7fb927bb1e5d-kube-api-access-pgnqf\") pod \"5e407739-e207-4224-954b-7fb927bb1e5d\" (UID: \"5e407739-e207-4224-954b-7fb927bb1e5d\") "
Sep 19 11:04:47.909949 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:47.909927    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e407739-e207-4224-954b-7fb927bb1e5d-kube-api-access-pgnqf" (OuterVolumeSpecName: "kube-api-access-pgnqf") pod "5e407739-e207-4224-954b-7fb927bb1e5d" (UID: "5e407739-e207-4224-954b-7fb927bb1e5d"). InnerVolumeSpecName "kube-api-access-pgnqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:04:48.009232 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:48.009142    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgnqf\" (UniqueName: \"kubernetes.io/projected/5e407739-e207-4224-954b-7fb927bb1e5d-kube-api-access-pgnqf\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:04:48.683818 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:48.683778    2576 generic.go:356] "Generic (PLEG): container finished" podID="5e407739-e207-4224-954b-7fb927bb1e5d" containerID="14e217fead784060a72ae9a703cfaf0388259201dae8e60696fbb4215699f530" exitCode=137
Sep 19 11:04:48.683818 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:48.683823    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7971-e3721d"
Sep 19 11:04:48.684102 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:48.683876    2576 scope.go:117] "RemoveContainer" containerID="14e217fead784060a72ae9a703cfaf0388259201dae8e60696fbb4215699f530"
Sep 19 11:04:48.693890 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:48.693869    2576 scope.go:117] "RemoveContainer" containerID="14e217fead784060a72ae9a703cfaf0388259201dae8e60696fbb4215699f530"
Sep 19 11:04:48.694182 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:04:48.694162    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14e217fead784060a72ae9a703cfaf0388259201dae8e60696fbb4215699f530\": container with ID starting with 14e217fead784060a72ae9a703cfaf0388259201dae8e60696fbb4215699f530 not found: ID does not exist" containerID="14e217fead784060a72ae9a703cfaf0388259201dae8e60696fbb4215699f530"
Sep 19 11:04:48.694261 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:48.694193    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14e217fead784060a72ae9a703cfaf0388259201dae8e60696fbb4215699f530"} err="failed to get container status \"14e217fead784060a72ae9a703cfaf0388259201dae8e60696fbb4215699f530\": rpc error: code = NotFound desc = could not find container \"14e217fead784060a72ae9a703cfaf0388259201dae8e60696fbb4215699f530\": container with ID starting with 14e217fead784060a72ae9a703cfaf0388259201dae8e60696fbb4215699f530 not found: ID does not exist"
Sep 19 11:04:48.694705 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:04:48.694680    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7971-e3721d\" is forbidden: User \"system:node:ip-10-0-142-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-165.ec2.internal' and this object" podUID="5e407739-e207-4224-954b-7fb927bb1e5d" pod="opendatahub/test-curl-7971-e3721d"
Sep 19 11:04:49.001469 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:04:49.001392    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e407739-e207-4224-954b-7fb927bb1e5d" path="/var/lib/kubelet/pods/5e407739-e207-4224-954b-7fb927bb1e5d/volumes"
Sep 19 11:06:07.345380 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.345300    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"]
Sep 19 11:06:07.345984 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.345782    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e407739-e207-4224-954b-7fb927bb1e5d" containerName="test-curl-7971-e3721d"
Sep 19 11:06:07.345984 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.345795    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e407739-e207-4224-954b-7fb927bb1e5d" containerName="test-curl-7971-e3721d"
Sep 19 11:06:07.345984 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.345881    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e407739-e207-4224-954b-7fb927bb1e5d" containerName="test-curl-7971-e3721d"
Sep 19 11:06:07.349185 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.349164    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"
Sep 19 11:06:07.351590 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.351568    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-b52037-serving-cert\""
Sep 19 11:06:07.360637 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.360601    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"]
Sep 19 11:06:07.413840 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.413819    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dc87\" (UniqueName: \"kubernetes.io/projected/81a466e0-48b3-4452-b24b-b7b5cf25ee13-kube-api-access-9dc87\") pod \"maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff\" (UID: \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"
Sep 19 11:06:07.413957 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.413852    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81a466e0-48b3-4452-b24b-b7b5cf25ee13-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff\" (UID: \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"
Sep 19 11:06:07.413957 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.413948    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a466e0-48b3-4452-b24b-b7b5cf25ee13-metrics-certs\") pod \"maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff\" (UID: \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"
Sep 19 11:06:07.414075 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.413995    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81a466e0-48b3-4452-b24b-b7b5cf25ee13-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff\" (UID: \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"
Sep 19 11:06:07.514523 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.514500    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dc87\" (UniqueName: \"kubernetes.io/projected/81a466e0-48b3-4452-b24b-b7b5cf25ee13-kube-api-access-9dc87\") pod \"maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff\" (UID: \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"
Sep 19 11:06:07.514629 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.514534    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81a466e0-48b3-4452-b24b-b7b5cf25ee13-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff\" (UID: \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"
Sep 19 11:06:07.514629 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.514586    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a466e0-48b3-4452-b24b-b7b5cf25ee13-metrics-certs\") pod \"maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff\" (UID: \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"
Sep 19 11:06:07.514761 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.514629    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81a466e0-48b3-4452-b24b-b7b5cf25ee13-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff\" (UID: \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"
Sep 19 11:06:07.515370 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.515302    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81a466e0-48b3-4452-b24b-b7b5cf25ee13-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff\" (UID: \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"
Sep 19 11:06:07.517096 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.517068    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81a466e0-48b3-4452-b24b-b7b5cf25ee13-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff\" (UID: \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"
Sep 19 11:06:07.517248 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.517231    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a466e0-48b3-4452-b24b-b7b5cf25ee13-metrics-certs\") pod \"maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff\" (UID: \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"
Sep 19 11:06:07.524542 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.524521    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dc87\" (UniqueName: \"kubernetes.io/projected/81a466e0-48b3-4452-b24b-b7b5cf25ee13-kube-api-access-9dc87\") pod \"maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff\" (UID: \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"
Sep 19 11:06:07.661536 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.661466    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"
Sep 19 11:06:07.933197 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.933176    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"]
Sep 19 11:06:07.935508 ip-10-0-142-165 kubenswrapper[2576]: W0919 11:06:07.935480    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81a466e0_48b3_4452_b24b_b7b5cf25ee13.slice/crio-22823d9d8e13b1c40a41e9e2b1a91dbd89861a5fbeef4116e848c1065169552a WatchSource:0}: Error finding container 22823d9d8e13b1c40a41e9e2b1a91dbd89861a5fbeef4116e848c1065169552a: Status 404 returned error can't find the container with id 22823d9d8e13b1c40a41e9e2b1a91dbd89861a5fbeef4116e848c1065169552a
Sep 19 11:06:07.936831 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:07.936812    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 11:06:08.111242 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:08.111209    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff" event={"ID":"81a466e0-48b3-4452-b24b-b7b5cf25ee13","Type":"ContainerStarted","Data":"22823d9d8e13b1c40a41e9e2b1a91dbd89861a5fbeef4116e848c1065169552a"}
Sep 19 11:06:09.118136 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:09.118104    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff" event={"ID":"81a466e0-48b3-4452-b24b-b7b5cf25ee13","Type":"ContainerStarted","Data":"864e17d7fa4b6bc663f2b8fde777ce2970ad8f874bac8573bc218aa13e1af63d"}
Sep 19 11:06:09.118521 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:09.118216    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"
Sep 19 11:06:09.135941 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:09.135872    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff" podStartSLOduration=1.701136156 podStartE2EDuration="2.135858782s" totalImagesPullingTime="434.722626ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:06:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:06:09.135155346 +0000 UTC m=+1778.631461437" watchObservedRunningTime="2026-09-19 11:06:09.135858782 +0000 UTC m=+1778.632164918"
Sep 19 11:06:15.128093 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:15.128037    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"
Sep 19 11:06:31.078956 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:31.078928    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 11:06:31.087097 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:06:31.087078    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 11:07:31.715308 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.715277    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"]
Sep 19 11:07:31.719116 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.719096    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.721619 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.721597    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-1b9c2b-openshift-default-dockercfg-76fbg\""
Sep 19 11:07:31.731400 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.731373    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"]
Sep 19 11:07:31.774015 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.773988    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95e41548-0b16-4415-aa2d-922806b657c5-istiod-ca-cert\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.774167 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.774076    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95e41548-0b16-4415-aa2d-922806b657c5-istio-token\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.774167 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.774132    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-istio-envoy\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.774245 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.774199    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-workload-socket\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.774245 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.774224    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95e41548-0b16-4415-aa2d-922806b657c5-istio-podinfo\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.774326 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.774245    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4lst\" (UniqueName: \"kubernetes.io/projected/95e41548-0b16-4415-aa2d-922806b657c5-kube-api-access-d4lst\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.774326 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.774262    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-istio-data\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.774326 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.774278    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-credential-socket\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.774326 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.774291    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-workload-certs\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.875192 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.875161    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95e41548-0b16-4415-aa2d-922806b657c5-istiod-ca-cert\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.875338 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.875203    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95e41548-0b16-4415-aa2d-922806b657c5-istio-token\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.875338 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.875245    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-istio-envoy\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.875338 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.875272    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-workload-socket\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.875338 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.875290    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95e41548-0b16-4415-aa2d-922806b657c5-istio-podinfo\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.875338 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.875313    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4lst\" (UniqueName: \"kubernetes.io/projected/95e41548-0b16-4415-aa2d-922806b657c5-kube-api-access-d4lst\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.875606 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.875339    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-istio-data\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.875606 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.875362    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-credential-socket\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.875606 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.875480    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-workload-certs\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.875781 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.875702    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-workload-socket\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.875861 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.875839    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-credential-socket\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.875913 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.875861    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-workload-certs\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.875978 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.875947    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-istio-data\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.876136 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.875977    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95e41548-0b16-4415-aa2d-922806b657c5-istiod-ca-cert\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.877856 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.877831    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-istio-envoy\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.877965 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.877950    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95e41548-0b16-4415-aa2d-922806b657c5-istio-podinfo\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.883709 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.883685    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95e41548-0b16-4415-aa2d-922806b657c5-istio-token\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:31.883839 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:31.883824    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4lst\" (UniqueName: \"kubernetes.io/projected/95e41548-0b16-4415-aa2d-922806b657c5-kube-api-access-d4lst\") pod \"e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") " pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:32.031525 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:32.031466    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:32.310782 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:32.310756    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"]
Sep 19 11:07:32.312438 ip-10-0-142-165 kubenswrapper[2576]: W0919 11:07:32.312413    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95e41548_0b16_4415_aa2d_922806b657c5.slice/crio-1e94c64c7d80ce605145302b2de8dc156934504bc888b9d3d43d10975e391c73 WatchSource:0}: Error finding container 1e94c64c7d80ce605145302b2de8dc156934504bc888b9d3d43d10975e391c73: Status 404 returned error can't find the container with id 1e94c64c7d80ce605145302b2de8dc156934504bc888b9d3d43d10975e391c73
Sep 19 11:07:32.314685 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:32.314654    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:07:32.314759 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:32.314714    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:07:32.314759 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:32.314740    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:07:32.558671 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:32.558631    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k" event={"ID":"95e41548-0b16-4415-aa2d-922806b657c5","Type":"ContainerStarted","Data":"7ab9fe2f5602c2b0a3c23db7f344075b3094c71f8ba8b04f86aeb377b7941a92"}
Sep 19 11:07:32.558859 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:32.558708    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k" event={"ID":"95e41548-0b16-4415-aa2d-922806b657c5","Type":"ContainerStarted","Data":"1e94c64c7d80ce605145302b2de8dc156934504bc888b9d3d43d10975e391c73"}
Sep 19 11:07:32.578619 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:32.578515    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k" podStartSLOduration=1.5785008619999998 podStartE2EDuration="1.578500862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:07:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:07:32.576410985 +0000 UTC m=+1862.072717094" watchObservedRunningTime="2026-09-19 11:07:32.578500862 +0000 UTC m=+1862.074806954"
Sep 19 11:07:33.031641 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:33.031604    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:33.037227 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:33.037200    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:33.564335 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:33.564309    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:33.565198 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:33.565180    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:37.304705 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:37.304669    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"]
Sep 19 11:07:37.309995 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:37.309973    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:07:37.312201 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:37.312181    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-1b9c2b-serving-cert\""
Sep 19 11:07:37.317658 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:37.317636    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"]
Sep 19 11:07:37.428580 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:37.428547    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp7f6\" (UniqueName: \"kubernetes.io/projected/dc608e31-f041-4f90-ae28-8e8932c9b2ca-kube-api-access-vp7f6\") pod \"maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g\" (UID: \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:07:37.428756 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:37.428679    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc608e31-f041-4f90-ae28-8e8932c9b2ca-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g\" (UID: \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:07:37.428756 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:37.428705    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc608e31-f041-4f90-ae28-8e8932c9b2ca-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g\" (UID: \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:07:37.428756 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:37.428743    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc608e31-f041-4f90-ae28-8e8932c9b2ca-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g\" (UID: \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:07:37.529459 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:37.529431    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp7f6\" (UniqueName: \"kubernetes.io/projected/dc608e31-f041-4f90-ae28-8e8932c9b2ca-kube-api-access-vp7f6\") pod \"maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g\" (UID: \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:07:37.529603 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:37.529495    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc608e31-f041-4f90-ae28-8e8932c9b2ca-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g\" (UID: \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:07:37.529603 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:37.529517    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc608e31-f041-4f90-ae28-8e8932c9b2ca-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g\" (UID: \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:07:37.529603 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:37.529539    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc608e31-f041-4f90-ae28-8e8932c9b2ca-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g\" (UID: \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:07:37.529766 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:07:37.529648    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-serving-cert: secret "maas-api-e2e-ipp-cleanup-1b9c2b-serving-cert" not found
Sep 19 11:07:37.529766 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:07:37.529709    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc608e31-f041-4f90-ae28-8e8932c9b2ca-maas-api-tls podName:dc608e31-f041-4f90-ae28-8e8932c9b2ca nodeName:}" failed. No retries permitted until 2026-09-19 11:07:38.029693162 +0000 UTC m=+1867.525999236 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dc608e31-f041-4f90-ae28-8e8932c9b2ca-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g" (UID: "dc608e31-f041-4f90-ae28-8e8932c9b2ca") : secret "maas-api-e2e-ipp-cleanup-1b9c2b-serving-cert" not found
Sep 19 11:07:37.530305 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:37.530281    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc608e31-f041-4f90-ae28-8e8932c9b2ca-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g\" (UID: \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:07:37.531996 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:37.531980    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc608e31-f041-4f90-ae28-8e8932c9b2ca-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g\" (UID: \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:07:37.540179 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:37.540160    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp7f6\" (UniqueName: \"kubernetes.io/projected/dc608e31-f041-4f90-ae28-8e8932c9b2ca-kube-api-access-vp7f6\") pod \"maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g\" (UID: \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:07:38.035960 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:38.035928    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc608e31-f041-4f90-ae28-8e8932c9b2ca-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g\" (UID: \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:07:38.036165 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:07:38.036085    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-serving-cert: secret "maas-api-e2e-ipp-cleanup-1b9c2b-serving-cert" not found
Sep 19 11:07:38.036231 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:07:38.036181    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc608e31-f041-4f90-ae28-8e8932c9b2ca-maas-api-tls podName:dc608e31-f041-4f90-ae28-8e8932c9b2ca nodeName:}" failed. No retries permitted until 2026-09-19 11:07:39.036154057 +0000 UTC m=+1868.532460150 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dc608e31-f041-4f90-ae28-8e8932c9b2ca-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g" (UID: "dc608e31-f041-4f90-ae28-8e8932c9b2ca") : secret "maas-api-e2e-ipp-cleanup-1b9c2b-serving-cert" not found
Sep 19 11:07:39.047544 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:39.047504    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc608e31-f041-4f90-ae28-8e8932c9b2ca-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g\" (UID: \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:07:39.050152 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:39.050126    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc608e31-f041-4f90-ae28-8e8932c9b2ca-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g\" (UID: \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:07:39.124976 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:39.124949    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:07:39.402011 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:39.401987    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"]
Sep 19 11:07:39.403990 ip-10-0-142-165 kubenswrapper[2576]: W0919 11:07:39.403963    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc608e31_f041_4f90_ae28_8e8932c9b2ca.slice/crio-de585fdf16ab90b1ffe2b50ebc91be332d94d01c4569e7550d6d810660130596 WatchSource:0}: Error finding container de585fdf16ab90b1ffe2b50ebc91be332d94d01c4569e7550d6d810660130596: Status 404 returned error can't find the container with id de585fdf16ab90b1ffe2b50ebc91be332d94d01c4569e7550d6d810660130596
Sep 19 11:07:39.597558 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:39.597523    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g" event={"ID":"dc608e31-f041-4f90-ae28-8e8932c9b2ca","Type":"ContainerStarted","Data":"de585fdf16ab90b1ffe2b50ebc91be332d94d01c4569e7550d6d810660130596"}
Sep 19 11:07:40.605193 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:40.605157    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g" event={"ID":"dc608e31-f041-4f90-ae28-8e8932c9b2ca","Type":"ContainerStarted","Data":"adf5be656d804e53dfcfb20dc51a1ed146ebddafce00149afa4a1561b5f1d463"}
Sep 19 11:07:40.605594 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:40.605295    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:07:40.623691 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:40.623646    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g" podStartSLOduration=3.26898314 podStartE2EDuration="3.623632544s" totalImagesPullingTime="354.649404ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:07:40.621118876 +0000 UTC m=+1870.117424971" watchObservedRunningTime="2026-09-19 11:07:40.623632544 +0000 UTC m=+1870.119938613"
Sep 19 11:07:46.615781 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:46.615749    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:07:54.690945 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:54.690908    2576 generic.go:356] "Generic (PLEG): container finished" podID="95e41548-0b16-4415-aa2d-922806b657c5" containerID="7ab9fe2f5602c2b0a3c23db7f344075b3094c71f8ba8b04f86aeb377b7941a92" exitCode=0
Sep 19 11:07:54.691375 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:54.690980    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k" event={"ID":"95e41548-0b16-4415-aa2d-922806b657c5","Type":"ContainerDied","Data":"7ab9fe2f5602c2b0a3c23db7f344075b3094c71f8ba8b04f86aeb377b7941a92"}
Sep 19 11:07:54.691432 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:54.691399    2576 scope.go:117] "RemoveContainer" containerID="7ab9fe2f5602c2b0a3c23db7f344075b3094c71f8ba8b04f86aeb377b7941a92"
Sep 19 11:07:54.692580 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:54.692552    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:07:54.692655 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:54.692623    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:07:54.692693 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:54.692658    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 11:07:55.705731 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:55.705697    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k" event={"ID":"95e41548-0b16-4415-aa2d-922806b657c5","Type":"ContainerStarted","Data":"14514e4f9f74dfea83804344421d1a5cf723a033b9cde9d55bc3109f21c6a824"}
Sep 19 11:07:56.032176 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:56.032087    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:07:56.032358 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:56.032330    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 19 11:07:56.032437 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:56.032390    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k" podUID="95e41548-0b16-4415-aa2d-922806b657c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 19 11:07:57.031821 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:57.031785    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 19 11:07:57.032190 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:57.031862    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k" podUID="95e41548-0b16-4415-aa2d-922806b657c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 19 11:07:58.032252 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:58.032213    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 19 11:07:58.032742 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:58.032267    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k" podUID="95e41548-0b16-4415-aa2d-922806b657c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 19 11:07:59.032093 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:59.032061    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 19 11:07:59.032263 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:07:59.032111    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k" podUID="95e41548-0b16-4415-aa2d-922806b657c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 19 11:08:00.032261 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:00.032223    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 19 11:08:00.032644 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:00.032283    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k" podUID="95e41548-0b16-4415-aa2d-922806b657c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 19 11:08:01.036115 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:01.036079    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:08:01.036572 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:01.036382    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:08:01.037036 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:01.037020    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:08:06.561180 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:06.561125    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"]
Sep 19 11:08:06.562000 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:06.561944    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g" podUID="dc608e31-f041-4f90-ae28-8e8932c9b2ca" containerName="maas-api" containerID="cri-o://adf5be656d804e53dfcfb20dc51a1ed146ebddafce00149afa4a1561b5f1d463" gracePeriod=30
Sep 19 11:08:06.611924 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:06.611885    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g" podUID="dc608e31-f041-4f90-ae28-8e8932c9b2ca" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.82:8443/health\": dial tcp 10.133.0.82:8443: connect: connection refused"
Sep 19 11:08:06.765517 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:06.765485    2576 generic.go:356] "Generic (PLEG): container finished" podID="dc608e31-f041-4f90-ae28-8e8932c9b2ca" containerID="adf5be656d804e53dfcfb20dc51a1ed146ebddafce00149afa4a1561b5f1d463" exitCode=0
Sep 19 11:08:06.765723 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:06.765549    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g" event={"ID":"dc608e31-f041-4f90-ae28-8e8932c9b2ca","Type":"ContainerDied","Data":"adf5be656d804e53dfcfb20dc51a1ed146ebddafce00149afa4a1561b5f1d463"}
Sep 19 11:08:06.827335 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:06.827280    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:08:06.921228 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:06.921200    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc608e31-f041-4f90-ae28-8e8932c9b2ca-maas-api-tls\") pod \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\" (UID: \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\") "
Sep 19 11:08:06.921228 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:06.921237    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vp7f6\" (UniqueName: \"kubernetes.io/projected/dc608e31-f041-4f90-ae28-8e8932c9b2ca-kube-api-access-vp7f6\") pod \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\" (UID: \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\") "
Sep 19 11:08:06.921449 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:06.921324    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc608e31-f041-4f90-ae28-8e8932c9b2ca-openshift-service-ca\") pod \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\" (UID: \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\") "
Sep 19 11:08:06.921449 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:06.921346    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc608e31-f041-4f90-ae28-8e8932c9b2ca-metrics-certs\") pod \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\" (UID: \"dc608e31-f041-4f90-ae28-8e8932c9b2ca\") "
Sep 19 11:08:06.921728 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:06.921692    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc608e31-f041-4f90-ae28-8e8932c9b2ca-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dc608e31-f041-4f90-ae28-8e8932c9b2ca" (UID: "dc608e31-f041-4f90-ae28-8e8932c9b2ca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:08:06.923527 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:06.923498    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc608e31-f041-4f90-ae28-8e8932c9b2ca-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "dc608e31-f041-4f90-ae28-8e8932c9b2ca" (UID: "dc608e31-f041-4f90-ae28-8e8932c9b2ca"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:08:06.923654 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:06.923550    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc608e31-f041-4f90-ae28-8e8932c9b2ca-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dc608e31-f041-4f90-ae28-8e8932c9b2ca" (UID: "dc608e31-f041-4f90-ae28-8e8932c9b2ca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:08:06.923707 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:06.923690    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc608e31-f041-4f90-ae28-8e8932c9b2ca-kube-api-access-vp7f6" (OuterVolumeSpecName: "kube-api-access-vp7f6") pod "dc608e31-f041-4f90-ae28-8e8932c9b2ca" (UID: "dc608e31-f041-4f90-ae28-8e8932c9b2ca"). InnerVolumeSpecName "kube-api-access-vp7f6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:08:07.022898 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:07.022865    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc608e31-f041-4f90-ae28-8e8932c9b2ca-openshift-service-ca\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:08:07.022898 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:07.022893    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc608e31-f041-4f90-ae28-8e8932c9b2ca-metrics-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:08:07.023092 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:07.022906    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc608e31-f041-4f90-ae28-8e8932c9b2ca-maas-api-tls\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:08:07.023092 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:07.022918    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vp7f6\" (UniqueName: \"kubernetes.io/projected/dc608e31-f041-4f90-ae28-8e8932c9b2ca-kube-api-access-vp7f6\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:08:07.773602 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:07.773572    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"
Sep 19 11:08:07.773602 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:07.773598    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g" event={"ID":"dc608e31-f041-4f90-ae28-8e8932c9b2ca","Type":"ContainerDied","Data":"de585fdf16ab90b1ffe2b50ebc91be332d94d01c4569e7550d6d810660130596"}
Sep 19 11:08:07.774120 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:07.773652    2576 scope.go:117] "RemoveContainer" containerID="adf5be656d804e53dfcfb20dc51a1ed146ebddafce00149afa4a1561b5f1d463"
Sep 19 11:08:07.792680 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:07.792654    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"]
Sep 19 11:08:07.796677 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:07.796654    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-1b9c2b-c8656777d-zcz2g"]
Sep 19 11:08:09.002086 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:09.002029    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc608e31-f041-4f90-ae28-8e8932c9b2ca" path="/var/lib/kubelet/pods/dc608e31-f041-4f90-ae28-8e8932c9b2ca/volumes"
Sep 19 11:08:12.952809 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:12.952775    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"]
Sep 19 11:08:12.953372 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:12.952993    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k" podUID="95e41548-0b16-4415-aa2d-922806b657c5" containerName="istio-proxy" containerID="cri-o://14514e4f9f74dfea83804344421d1a5cf723a033b9cde9d55bc3109f21c6a824" gracePeriod=30
Sep 19 11:08:16.038868 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:16.038838    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 11:08:16.039256 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:16.038896    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k" podUID="95e41548-0b16-4415-aa2d-922806b657c5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 11:08:18.206993 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.206964    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:08:18.219813 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.219758    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95e41548-0b16-4415-aa2d-922806b657c5-istio-token\") pod \"95e41548-0b16-4415-aa2d-922806b657c5\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") "
Sep 19 11:08:18.219813 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.219784    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-istio-envoy\") pod \"95e41548-0b16-4415-aa2d-922806b657c5\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") "
Sep 19 11:08:18.219972 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.219852    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95e41548-0b16-4415-aa2d-922806b657c5-istiod-ca-cert\") pod \"95e41548-0b16-4415-aa2d-922806b657c5\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") "
Sep 19 11:08:18.219972 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.219874    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4lst\" (UniqueName: \"kubernetes.io/projected/95e41548-0b16-4415-aa2d-922806b657c5-kube-api-access-d4lst\") pod \"95e41548-0b16-4415-aa2d-922806b657c5\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") "
Sep 19 11:08:18.219972 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.219898    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95e41548-0b16-4415-aa2d-922806b657c5-istio-podinfo\") pod \"95e41548-0b16-4415-aa2d-922806b657c5\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") "
Sep 19 11:08:18.219972 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.219914    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-credential-socket\") pod \"95e41548-0b16-4415-aa2d-922806b657c5\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") "
Sep 19 11:08:18.219972 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.219947    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-workload-socket\") pod \"95e41548-0b16-4415-aa2d-922806b657c5\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") "
Sep 19 11:08:18.220256 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.220005    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-workload-certs\") pod \"95e41548-0b16-4415-aa2d-922806b657c5\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") "
Sep 19 11:08:18.220256 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.220037    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-istio-data\") pod \"95e41548-0b16-4415-aa2d-922806b657c5\" (UID: \"95e41548-0b16-4415-aa2d-922806b657c5\") "
Sep 19 11:08:18.220369 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.220303    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95e41548-0b16-4415-aa2d-922806b657c5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "95e41548-0b16-4415-aa2d-922806b657c5" (UID: "95e41548-0b16-4415-aa2d-922806b657c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:08:18.220478 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.220426    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "95e41548-0b16-4415-aa2d-922806b657c5" (UID: "95e41548-0b16-4415-aa2d-922806b657c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:08:18.220611 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.220500    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "95e41548-0b16-4415-aa2d-922806b657c5" (UID: "95e41548-0b16-4415-aa2d-922806b657c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:08:18.220788 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.220766    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "95e41548-0b16-4415-aa2d-922806b657c5" (UID: "95e41548-0b16-4415-aa2d-922806b657c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:08:18.220837 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.220743    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-istio-data" (OuterVolumeSpecName: "istio-data") pod "95e41548-0b16-4415-aa2d-922806b657c5" (UID: "95e41548-0b16-4415-aa2d-922806b657c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:08:18.222090 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.222064    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95e41548-0b16-4415-aa2d-922806b657c5-istio-token" (OuterVolumeSpecName: "istio-token") pod "95e41548-0b16-4415-aa2d-922806b657c5" (UID: "95e41548-0b16-4415-aa2d-922806b657c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:08:18.222201 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.222154    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/95e41548-0b16-4415-aa2d-922806b657c5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "95e41548-0b16-4415-aa2d-922806b657c5" (UID: "95e41548-0b16-4415-aa2d-922806b657c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 11:08:18.222621 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.222602    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "95e41548-0b16-4415-aa2d-922806b657c5" (UID: "95e41548-0b16-4415-aa2d-922806b657c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:08:18.222686 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.222605    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95e41548-0b16-4415-aa2d-922806b657c5-kube-api-access-d4lst" (OuterVolumeSpecName: "kube-api-access-d4lst") pod "95e41548-0b16-4415-aa2d-922806b657c5" (UID: "95e41548-0b16-4415-aa2d-922806b657c5"). InnerVolumeSpecName "kube-api-access-d4lst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:08:18.321670 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.321637    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:08:18.321670 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.321669    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:08:18.321847 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.321681    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95e41548-0b16-4415-aa2d-922806b657c5-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:08:18.321847 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.321689    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:08:18.321847 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.321697    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95e41548-0b16-4415-aa2d-922806b657c5-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:08:18.321847 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.321705    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4lst\" (UniqueName: \"kubernetes.io/projected/95e41548-0b16-4415-aa2d-922806b657c5-kube-api-access-d4lst\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:08:18.321847 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.321715    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95e41548-0b16-4415-aa2d-922806b657c5-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:08:18.321847 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.321723    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:08:18.321847 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.321732    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95e41548-0b16-4415-aa2d-922806b657c5-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:08:18.835004 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.834972    2576 generic.go:356] "Generic (PLEG): container finished" podID="95e41548-0b16-4415-aa2d-922806b657c5" containerID="14514e4f9f74dfea83804344421d1a5cf723a033b9cde9d55bc3109f21c6a824" exitCode=0
Sep 19 11:08:18.835214 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.835012    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k" event={"ID":"95e41548-0b16-4415-aa2d-922806b657c5","Type":"ContainerDied","Data":"14514e4f9f74dfea83804344421d1a5cf723a033b9cde9d55bc3109f21c6a824"}
Sep 19 11:08:18.835214 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.835019    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"
Sep 19 11:08:18.835214 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.835069    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k" event={"ID":"95e41548-0b16-4415-aa2d-922806b657c5","Type":"ContainerDied","Data":"1e94c64c7d80ce605145302b2de8dc156934504bc888b9d3d43d10975e391c73"}
Sep 19 11:08:18.835214 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.835091    2576 scope.go:117] "RemoveContainer" containerID="14514e4f9f74dfea83804344421d1a5cf723a033b9cde9d55bc3109f21c6a824"
Sep 19 11:08:18.845543 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.845521    2576 scope.go:117] "RemoveContainer" containerID="7ab9fe2f5602c2b0a3c23db7f344075b3094c71f8ba8b04f86aeb377b7941a92"
Sep 19 11:08:18.854534 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.854508    2576 scope.go:117] "RemoveContainer" containerID="14514e4f9f74dfea83804344421d1a5cf723a033b9cde9d55bc3109f21c6a824"
Sep 19 11:08:18.854882 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:08:18.854859    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14514e4f9f74dfea83804344421d1a5cf723a033b9cde9d55bc3109f21c6a824\": container with ID starting with 14514e4f9f74dfea83804344421d1a5cf723a033b9cde9d55bc3109f21c6a824 not found: ID does not exist" containerID="14514e4f9f74dfea83804344421d1a5cf723a033b9cde9d55bc3109f21c6a824"
Sep 19 11:08:18.854990 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.854892    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14514e4f9f74dfea83804344421d1a5cf723a033b9cde9d55bc3109f21c6a824"} err="failed to get container status \"14514e4f9f74dfea83804344421d1a5cf723a033b9cde9d55bc3109f21c6a824\": rpc error: code = NotFound desc = could not find container \"14514e4f9f74dfea83804344421d1a5cf723a033b9cde9d55bc3109f21c6a824\": container with ID starting with 14514e4f9f74dfea83804344421d1a5cf723a033b9cde9d55bc3109f21c6a824 not found: ID does not exist"
Sep 19 11:08:18.854990 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.854916    2576 scope.go:117] "RemoveContainer" containerID="7ab9fe2f5602c2b0a3c23db7f344075b3094c71f8ba8b04f86aeb377b7941a92"
Sep 19 11:08:18.855367 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:08:18.855348    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ab9fe2f5602c2b0a3c23db7f344075b3094c71f8ba8b04f86aeb377b7941a92\": container with ID starting with 7ab9fe2f5602c2b0a3c23db7f344075b3094c71f8ba8b04f86aeb377b7941a92 not found: ID does not exist" containerID="7ab9fe2f5602c2b0a3c23db7f344075b3094c71f8ba8b04f86aeb377b7941a92"
Sep 19 11:08:18.855427 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.855375    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ab9fe2f5602c2b0a3c23db7f344075b3094c71f8ba8b04f86aeb377b7941a92"} err="failed to get container status \"7ab9fe2f5602c2b0a3c23db7f344075b3094c71f8ba8b04f86aeb377b7941a92\": rpc error: code = NotFound desc = could not find container \"7ab9fe2f5602c2b0a3c23db7f344075b3094c71f8ba8b04f86aeb377b7941a92\": container with ID starting with 7ab9fe2f5602c2b0a3c23db7f344075b3094c71f8ba8b04f86aeb377b7941a92 not found: ID does not exist"
Sep 19 11:08:18.857850 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.857829    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"]
Sep 19 11:08:18.859976 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:18.859954    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-1b9c2b-openshift-default-98696db86-wr64k"]
Sep 19 11:08:19.001939 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:19.001912    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95e41548-0b16-4415-aa2d-922806b657c5" path="/var/lib/kubelet/pods/95e41548-0b16-4415-aa2d-922806b657c5/volumes"
Sep 19 11:08:32.058522 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.058491    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"]
Sep 19 11:08:32.059059 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.058709    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff" podUID="81a466e0-48b3-4452-b24b-b7b5cf25ee13" containerName="maas-api" containerID="cri-o://864e17d7fa4b6bc663f2b8fde777ce2970ad8f874bac8573bc218aa13e1af63d" gracePeriod=30
Sep 19 11:08:32.313673 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.313609    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"
Sep 19 11:08:32.439843 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.439806    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81a466e0-48b3-4452-b24b-b7b5cf25ee13-maas-api-tls\") pod \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\" (UID: \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\") "
Sep 19 11:08:32.440021 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.439874    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dc87\" (UniqueName: \"kubernetes.io/projected/81a466e0-48b3-4452-b24b-b7b5cf25ee13-kube-api-access-9dc87\") pod \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\" (UID: \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\") "
Sep 19 11:08:32.440021 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.439946    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a466e0-48b3-4452-b24b-b7b5cf25ee13-metrics-certs\") pod \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\" (UID: \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\") "
Sep 19 11:08:32.440021 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.440002    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81a466e0-48b3-4452-b24b-b7b5cf25ee13-openshift-service-ca\") pod \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\" (UID: \"81a466e0-48b3-4452-b24b-b7b5cf25ee13\") "
Sep 19 11:08:32.441711 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.440354    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81a466e0-48b3-4452-b24b-b7b5cf25ee13-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "81a466e0-48b3-4452-b24b-b7b5cf25ee13" (UID: "81a466e0-48b3-4452-b24b-b7b5cf25ee13"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:08:32.443273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.443245    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81a466e0-48b3-4452-b24b-b7b5cf25ee13-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "81a466e0-48b3-4452-b24b-b7b5cf25ee13" (UID: "81a466e0-48b3-4452-b24b-b7b5cf25ee13"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:08:32.443273 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.443254    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81a466e0-48b3-4452-b24b-b7b5cf25ee13-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "81a466e0-48b3-4452-b24b-b7b5cf25ee13" (UID: "81a466e0-48b3-4452-b24b-b7b5cf25ee13"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:08:32.443532 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.443514    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81a466e0-48b3-4452-b24b-b7b5cf25ee13-kube-api-access-9dc87" (OuterVolumeSpecName: "kube-api-access-9dc87") pod "81a466e0-48b3-4452-b24b-b7b5cf25ee13" (UID: "81a466e0-48b3-4452-b24b-b7b5cf25ee13"). InnerVolumeSpecName "kube-api-access-9dc87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:08:32.541871 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.541844    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dc87\" (UniqueName: \"kubernetes.io/projected/81a466e0-48b3-4452-b24b-b7b5cf25ee13-kube-api-access-9dc87\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:08:32.541871 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.541870    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a466e0-48b3-4452-b24b-b7b5cf25ee13-metrics-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:08:32.541871 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.541879    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81a466e0-48b3-4452-b24b-b7b5cf25ee13-openshift-service-ca\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:08:32.542179 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.541888    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81a466e0-48b3-4452-b24b-b7b5cf25ee13-maas-api-tls\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:08:32.910564 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.910530    2576 generic.go:356] "Generic (PLEG): container finished" podID="81a466e0-48b3-4452-b24b-b7b5cf25ee13" containerID="864e17d7fa4b6bc663f2b8fde777ce2970ad8f874bac8573bc218aa13e1af63d" exitCode=0
Sep 19 11:08:32.910749 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.910573    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"
Sep 19 11:08:32.910749 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.910616    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff" event={"ID":"81a466e0-48b3-4452-b24b-b7b5cf25ee13","Type":"ContainerDied","Data":"864e17d7fa4b6bc663f2b8fde777ce2970ad8f874bac8573bc218aa13e1af63d"}
Sep 19 11:08:32.910749 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.910674    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff" event={"ID":"81a466e0-48b3-4452-b24b-b7b5cf25ee13","Type":"ContainerDied","Data":"22823d9d8e13b1c40a41e9e2b1a91dbd89861a5fbeef4116e848c1065169552a"}
Sep 19 11:08:32.910749 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.910704    2576 scope.go:117] "RemoveContainer" containerID="864e17d7fa4b6bc663f2b8fde777ce2970ad8f874bac8573bc218aa13e1af63d"
Sep 19 11:08:32.920544 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.920521    2576 scope.go:117] "RemoveContainer" containerID="864e17d7fa4b6bc663f2b8fde777ce2970ad8f874bac8573bc218aa13e1af63d"
Sep 19 11:08:32.920795 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:08:32.920778    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"864e17d7fa4b6bc663f2b8fde777ce2970ad8f874bac8573bc218aa13e1af63d\": container with ID starting with 864e17d7fa4b6bc663f2b8fde777ce2970ad8f874bac8573bc218aa13e1af63d not found: ID does not exist" containerID="864e17d7fa4b6bc663f2b8fde777ce2970ad8f874bac8573bc218aa13e1af63d"
Sep 19 11:08:32.920842 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.920804    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"864e17d7fa4b6bc663f2b8fde777ce2970ad8f874bac8573bc218aa13e1af63d"} err="failed to get container status \"864e17d7fa4b6bc663f2b8fde777ce2970ad8f874bac8573bc218aa13e1af63d\": rpc error: code = NotFound desc = could not find container \"864e17d7fa4b6bc663f2b8fde777ce2970ad8f874bac8573bc218aa13e1af63d\": container with ID starting with 864e17d7fa4b6bc663f2b8fde777ce2970ad8f874bac8573bc218aa13e1af63d not found: ID does not exist"
Sep 19 11:08:32.931763 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.931738    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"]
Sep 19 11:08:32.934987 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:32.934968    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b52037-ffc6444dc-vwwff"]
Sep 19 11:08:33.001733 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:08:33.001710    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81a466e0-48b3-4452-b24b-b7b5cf25ee13" path="/var/lib/kubelet/pods/81a466e0-48b3-4452-b24b-b7b5cf25ee13/volumes"
Sep 19 11:09:54.212862 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:54.212828    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"]
Sep 19 11:09:54.213623 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:54.213106    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4" podUID="26a85e00-629b-4d45-8984-377f1e13123b" containerName="maas-api" containerID="cri-o://3f0cc357162ff0f15e0828e445bffa96e8ad4302bb7cb7bdb9b67cd78741ab81" gracePeriod=30
Sep 19 11:09:54.336997 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:54.336969    2576 generic.go:356] "Generic (PLEG): container finished" podID="26a85e00-629b-4d45-8984-377f1e13123b" containerID="3f0cc357162ff0f15e0828e445bffa96e8ad4302bb7cb7bdb9b67cd78741ab81" exitCode=0
Sep 19 11:09:54.337195 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:54.337075    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4" event={"ID":"26a85e00-629b-4d45-8984-377f1e13123b","Type":"ContainerDied","Data":"3f0cc357162ff0f15e0828e445bffa96e8ad4302bb7cb7bdb9b67cd78741ab81"}
Sep 19 11:09:54.477835 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:54.477772    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"
Sep 19 11:09:54.671887 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:54.671854    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26a85e00-629b-4d45-8984-377f1e13123b-maas-api-tls\") pod \"26a85e00-629b-4d45-8984-377f1e13123b\" (UID: \"26a85e00-629b-4d45-8984-377f1e13123b\") "
Sep 19 11:09:54.672082 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:54.671911    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26a85e00-629b-4d45-8984-377f1e13123b-metrics-certs\") pod \"26a85e00-629b-4d45-8984-377f1e13123b\" (UID: \"26a85e00-629b-4d45-8984-377f1e13123b\") "
Sep 19 11:09:54.672082 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:54.671982    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22gg9\" (UniqueName: \"kubernetes.io/projected/26a85e00-629b-4d45-8984-377f1e13123b-kube-api-access-22gg9\") pod \"26a85e00-629b-4d45-8984-377f1e13123b\" (UID: \"26a85e00-629b-4d45-8984-377f1e13123b\") "
Sep 19 11:09:54.672082 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:54.672018    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26a85e00-629b-4d45-8984-377f1e13123b-openshift-service-ca\") pod \"26a85e00-629b-4d45-8984-377f1e13123b\" (UID: \"26a85e00-629b-4d45-8984-377f1e13123b\") "
Sep 19 11:09:54.672487 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:54.672455    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26a85e00-629b-4d45-8984-377f1e13123b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "26a85e00-629b-4d45-8984-377f1e13123b" (UID: "26a85e00-629b-4d45-8984-377f1e13123b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:09:54.674246 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:54.674214    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26a85e00-629b-4d45-8984-377f1e13123b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "26a85e00-629b-4d45-8984-377f1e13123b" (UID: "26a85e00-629b-4d45-8984-377f1e13123b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:09:54.674358 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:54.674239    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26a85e00-629b-4d45-8984-377f1e13123b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "26a85e00-629b-4d45-8984-377f1e13123b" (UID: "26a85e00-629b-4d45-8984-377f1e13123b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:09:54.674358 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:54.674279    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26a85e00-629b-4d45-8984-377f1e13123b-kube-api-access-22gg9" (OuterVolumeSpecName: "kube-api-access-22gg9") pod "26a85e00-629b-4d45-8984-377f1e13123b" (UID: "26a85e00-629b-4d45-8984-377f1e13123b"). InnerVolumeSpecName "kube-api-access-22gg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:09:54.773724 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:54.773653    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/26a85e00-629b-4d45-8984-377f1e13123b-maas-api-tls\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:09:54.773724 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:54.773679    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26a85e00-629b-4d45-8984-377f1e13123b-metrics-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:09:54.773724 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:54.773688    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22gg9\" (UniqueName: \"kubernetes.io/projected/26a85e00-629b-4d45-8984-377f1e13123b-kube-api-access-22gg9\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:09:54.773724 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:54.773696    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/26a85e00-629b-4d45-8984-377f1e13123b-openshift-service-ca\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:09:55.343895 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:55.343867    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"
Sep 19 11:09:55.344375 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:55.343867    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4" event={"ID":"26a85e00-629b-4d45-8984-377f1e13123b","Type":"ContainerDied","Data":"34932a526d32f627e99cb9e6a0fe9068cac2117834f3b568a67abb65f1753989"}
Sep 19 11:09:55.344375 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:55.344062    2576 scope.go:117] "RemoveContainer" containerID="3f0cc357162ff0f15e0828e445bffa96e8ad4302bb7cb7bdb9b67cd78741ab81"
Sep 19 11:09:55.361557 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:55.361534    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"]
Sep 19 11:09:55.364809 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:55.364779    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-137f88-5b6cfbb55d-qghm4"]
Sep 19 11:09:57.001624 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:09:57.001591    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26a85e00-629b-4d45-8984-377f1e13123b" path="/var/lib/kubelet/pods/26a85e00-629b-4d45-8984-377f1e13123b/volumes"
Sep 19 11:10:00.841465 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:00.841430    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"]
Sep 19 11:10:00.841918 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:00.841685    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l" podUID="3142ef93-54d6-463d-985f-6fb7b28eadfe" containerName="istio-proxy" containerID="cri-o://f4e78a4853f820c602f6772be9530807b335834a955d18c072056e1b3ecc0013" gracePeriod=30
Sep 19 11:10:06.092644 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.092619    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 11:10:06.166143 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.166070    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-workload-socket\") pod \"3142ef93-54d6-463d-985f-6fb7b28eadfe\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") "
Sep 19 11:10:06.166143 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.166103    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-envoy\") pod \"3142ef93-54d6-463d-985f-6fb7b28eadfe\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") "
Sep 19 11:10:06.166143 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.166127    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-token\") pod \"3142ef93-54d6-463d-985f-6fb7b28eadfe\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") "
Sep 19 11:10:06.166344 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.166195    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-podinfo\") pod \"3142ef93-54d6-463d-985f-6fb7b28eadfe\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") "
Sep 19 11:10:06.166389 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.166342    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3142ef93-54d6-463d-985f-6fb7b28eadfe" (UID: "3142ef93-54d6-463d-985f-6fb7b28eadfe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:10:06.166389 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.166375    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-data\") pod \"3142ef93-54d6-463d-985f-6fb7b28eadfe\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") "
Sep 19 11:10:06.166496 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.166433    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3142ef93-54d6-463d-985f-6fb7b28eadfe-istiod-ca-cert\") pod \"3142ef93-54d6-463d-985f-6fb7b28eadfe\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") "
Sep 19 11:10:06.166496 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.166471    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-workload-certs\") pod \"3142ef93-54d6-463d-985f-6fb7b28eadfe\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") "
Sep 19 11:10:06.166606 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.166548    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rntr8\" (UniqueName: \"kubernetes.io/projected/3142ef93-54d6-463d-985f-6fb7b28eadfe-kube-api-access-rntr8\") pod \"3142ef93-54d6-463d-985f-6fb7b28eadfe\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") "
Sep 19 11:10:06.166606 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.166588    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-credential-socket\") pod \"3142ef93-54d6-463d-985f-6fb7b28eadfe\" (UID: \"3142ef93-54d6-463d-985f-6fb7b28eadfe\") "
Sep 19 11:10:06.166722 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.166606    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-data" (OuterVolumeSpecName: "istio-data") pod "3142ef93-54d6-463d-985f-6fb7b28eadfe" (UID: "3142ef93-54d6-463d-985f-6fb7b28eadfe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:10:06.166781 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.166750    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3142ef93-54d6-463d-985f-6fb7b28eadfe" (UID: "3142ef93-54d6-463d-985f-6fb7b28eadfe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:10:06.166857 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.166833    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3142ef93-54d6-463d-985f-6fb7b28eadfe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3142ef93-54d6-463d-985f-6fb7b28eadfe" (UID: "3142ef93-54d6-463d-985f-6fb7b28eadfe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:10:06.167022 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.167005    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:06.167108 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.167028    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3142ef93-54d6-463d-985f-6fb7b28eadfe-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:06.167108 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.167065    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:06.167214 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.167080    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:06.167214 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.167146    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3142ef93-54d6-463d-985f-6fb7b28eadfe" (UID: "3142ef93-54d6-463d-985f-6fb7b28eadfe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:10:06.168521 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.168499    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-token" (OuterVolumeSpecName: "istio-token") pod "3142ef93-54d6-463d-985f-6fb7b28eadfe" (UID: "3142ef93-54d6-463d-985f-6fb7b28eadfe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:10:06.168999 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.168976    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3142ef93-54d6-463d-985f-6fb7b28eadfe" (UID: "3142ef93-54d6-463d-985f-6fb7b28eadfe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:10:06.168999 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.168989    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3142ef93-54d6-463d-985f-6fb7b28eadfe" (UID: "3142ef93-54d6-463d-985f-6fb7b28eadfe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 11:10:06.168999 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.169003    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3142ef93-54d6-463d-985f-6fb7b28eadfe-kube-api-access-rntr8" (OuterVolumeSpecName: "kube-api-access-rntr8") pod "3142ef93-54d6-463d-985f-6fb7b28eadfe" (UID: "3142ef93-54d6-463d-985f-6fb7b28eadfe"). InnerVolumeSpecName "kube-api-access-rntr8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:10:06.267795 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.267762    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:06.267795 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.267790    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rntr8\" (UniqueName: \"kubernetes.io/projected/3142ef93-54d6-463d-985f-6fb7b28eadfe-kube-api-access-rntr8\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:06.267962 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.267804    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:06.267962 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.267817    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:06.267962 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.267828    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3142ef93-54d6-463d-985f-6fb7b28eadfe-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:06.416895 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.416822    2576 generic.go:356] "Generic (PLEG): container finished" podID="3142ef93-54d6-463d-985f-6fb7b28eadfe" containerID="f4e78a4853f820c602f6772be9530807b335834a955d18c072056e1b3ecc0013" exitCode=0
Sep 19 11:10:06.416895 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.416867    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"
Sep 19 11:10:06.417073 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.416900    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l" event={"ID":"3142ef93-54d6-463d-985f-6fb7b28eadfe","Type":"ContainerDied","Data":"f4e78a4853f820c602f6772be9530807b335834a955d18c072056e1b3ecc0013"}
Sep 19 11:10:06.417073 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.416952    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l" event={"ID":"3142ef93-54d6-463d-985f-6fb7b28eadfe","Type":"ContainerDied","Data":"5689f91bd590219498fd5b6e99bbf730e858a7eede8c5d02298e786495a19543"}
Sep 19 11:10:06.417073 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.416977    2576 scope.go:117] "RemoveContainer" containerID="f4e78a4853f820c602f6772be9530807b335834a955d18c072056e1b3ecc0013"
Sep 19 11:10:06.427115 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.427105    2576 scope.go:117] "RemoveContainer" containerID="67bacaa6732578b422d3b1e7ce6a4c8d59cc42cd93c26be2e176951431fb8b27"
Sep 19 11:10:06.435482 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.435439    2576 scope.go:117] "RemoveContainer" containerID="f4e78a4853f820c602f6772be9530807b335834a955d18c072056e1b3ecc0013"
Sep 19 11:10:06.436205 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:10:06.436178    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4e78a4853f820c602f6772be9530807b335834a955d18c072056e1b3ecc0013\": container with ID starting with f4e78a4853f820c602f6772be9530807b335834a955d18c072056e1b3ecc0013 not found: ID does not exist" containerID="f4e78a4853f820c602f6772be9530807b335834a955d18c072056e1b3ecc0013"
Sep 19 11:10:06.436322 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.436214    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4e78a4853f820c602f6772be9530807b335834a955d18c072056e1b3ecc0013"} err="failed to get container status \"f4e78a4853f820c602f6772be9530807b335834a955d18c072056e1b3ecc0013\": rpc error: code = NotFound desc = could not find container \"f4e78a4853f820c602f6772be9530807b335834a955d18c072056e1b3ecc0013\": container with ID starting with f4e78a4853f820c602f6772be9530807b335834a955d18c072056e1b3ecc0013 not found: ID does not exist"
Sep 19 11:10:06.436322 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.436236    2576 scope.go:117] "RemoveContainer" containerID="67bacaa6732578b422d3b1e7ce6a4c8d59cc42cd93c26be2e176951431fb8b27"
Sep 19 11:10:06.436571 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:10:06.436553    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67bacaa6732578b422d3b1e7ce6a4c8d59cc42cd93c26be2e176951431fb8b27\": container with ID starting with 67bacaa6732578b422d3b1e7ce6a4c8d59cc42cd93c26be2e176951431fb8b27 not found: ID does not exist" containerID="67bacaa6732578b422d3b1e7ce6a4c8d59cc42cd93c26be2e176951431fb8b27"
Sep 19 11:10:06.436637 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.436580    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67bacaa6732578b422d3b1e7ce6a4c8d59cc42cd93c26be2e176951431fb8b27"} err="failed to get container status \"67bacaa6732578b422d3b1e7ce6a4c8d59cc42cd93c26be2e176951431fb8b27\": rpc error: code = NotFound desc = could not find container \"67bacaa6732578b422d3b1e7ce6a4c8d59cc42cd93c26be2e176951431fb8b27\": container with ID starting with 67bacaa6732578b422d3b1e7ce6a4c8d59cc42cd93c26be2e176951431fb8b27 not found: ID does not exist"
Sep 19 11:10:06.438689 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.438671    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"]
Sep 19 11:10:06.443491 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:06.443469    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-137f88-openshift-default-549d4696c4-48r4l"]
Sep 19 11:10:07.001870 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:07.001841    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3142ef93-54d6-463d-985f-6fb7b28eadfe" path="/var/lib/kubelet/pods/3142ef93-54d6-463d-985f-6fb7b28eadfe/volumes"
Sep 19 11:10:11.696184 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:11.696147    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"]
Sep 19 11:10:11.696576 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:11.696353    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz" podUID="0933555e-5813-45a1-bbbd-0bd17362423b" containerName="maas-api" containerID="cri-o://6c1f37bbb573e63a908aea9d3b4362e92bdaa58d584709011ee619bf5d699425" gracePeriod=30
Sep 19 11:10:11.952539 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:11.952473    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"
Sep 19 11:10:12.015734 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.015707    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0933555e-5813-45a1-bbbd-0bd17362423b-metrics-certs\") pod \"0933555e-5813-45a1-bbbd-0bd17362423b\" (UID: \"0933555e-5813-45a1-bbbd-0bd17362423b\") "
Sep 19 11:10:12.015935 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.015750    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0933555e-5813-45a1-bbbd-0bd17362423b-openshift-service-ca\") pod \"0933555e-5813-45a1-bbbd-0bd17362423b\" (UID: \"0933555e-5813-45a1-bbbd-0bd17362423b\") "
Sep 19 11:10:12.015935 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.015846    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdww6\" (UniqueName: \"kubernetes.io/projected/0933555e-5813-45a1-bbbd-0bd17362423b-kube-api-access-gdww6\") pod \"0933555e-5813-45a1-bbbd-0bd17362423b\" (UID: \"0933555e-5813-45a1-bbbd-0bd17362423b\") "
Sep 19 11:10:12.015935 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.015887    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0933555e-5813-45a1-bbbd-0bd17362423b-maas-api-tls\") pod \"0933555e-5813-45a1-bbbd-0bd17362423b\" (UID: \"0933555e-5813-45a1-bbbd-0bd17362423b\") "
Sep 19 11:10:12.016530 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.016219    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0933555e-5813-45a1-bbbd-0bd17362423b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0933555e-5813-45a1-bbbd-0bd17362423b" (UID: "0933555e-5813-45a1-bbbd-0bd17362423b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:10:12.018314 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.018276    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0933555e-5813-45a1-bbbd-0bd17362423b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0933555e-5813-45a1-bbbd-0bd17362423b" (UID: "0933555e-5813-45a1-bbbd-0bd17362423b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:10:12.018429 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.018360    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0933555e-5813-45a1-bbbd-0bd17362423b-kube-api-access-gdww6" (OuterVolumeSpecName: "kube-api-access-gdww6") pod "0933555e-5813-45a1-bbbd-0bd17362423b" (UID: "0933555e-5813-45a1-bbbd-0bd17362423b"). InnerVolumeSpecName "kube-api-access-gdww6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:10:12.018718 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.018691    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0933555e-5813-45a1-bbbd-0bd17362423b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0933555e-5813-45a1-bbbd-0bd17362423b" (UID: "0933555e-5813-45a1-bbbd-0bd17362423b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:10:12.117024 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.116981    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0933555e-5813-45a1-bbbd-0bd17362423b-metrics-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:12.117024 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.117016    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0933555e-5813-45a1-bbbd-0bd17362423b-openshift-service-ca\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:12.117024 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.117025    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdww6\" (UniqueName: \"kubernetes.io/projected/0933555e-5813-45a1-bbbd-0bd17362423b-kube-api-access-gdww6\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:12.117024 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.117034    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0933555e-5813-45a1-bbbd-0bd17362423b-maas-api-tls\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:12.453890 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.453859    2576 generic.go:356] "Generic (PLEG): container finished" podID="0933555e-5813-45a1-bbbd-0bd17362423b" containerID="6c1f37bbb573e63a908aea9d3b4362e92bdaa58d584709011ee619bf5d699425" exitCode=0
Sep 19 11:10:12.454138 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.453911    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"
Sep 19 11:10:12.454138 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.453924    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz" event={"ID":"0933555e-5813-45a1-bbbd-0bd17362423b","Type":"ContainerDied","Data":"6c1f37bbb573e63a908aea9d3b4362e92bdaa58d584709011ee619bf5d699425"}
Sep 19 11:10:12.454138 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.453970    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz" event={"ID":"0933555e-5813-45a1-bbbd-0bd17362423b","Type":"ContainerDied","Data":"52923eb4199294d544530bfbc6d339ca715cd6c990192a67a2b3fb9421dc67e9"}
Sep 19 11:10:12.454138 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.453991    2576 scope.go:117] "RemoveContainer" containerID="6c1f37bbb573e63a908aea9d3b4362e92bdaa58d584709011ee619bf5d699425"
Sep 19 11:10:12.463868 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.463854    2576 scope.go:117] "RemoveContainer" containerID="6c1f37bbb573e63a908aea9d3b4362e92bdaa58d584709011ee619bf5d699425"
Sep 19 11:10:12.464171 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:10:12.464152    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c1f37bbb573e63a908aea9d3b4362e92bdaa58d584709011ee619bf5d699425\": container with ID starting with 6c1f37bbb573e63a908aea9d3b4362e92bdaa58d584709011ee619bf5d699425 not found: ID does not exist" containerID="6c1f37bbb573e63a908aea9d3b4362e92bdaa58d584709011ee619bf5d699425"
Sep 19 11:10:12.464245 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.464177    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c1f37bbb573e63a908aea9d3b4362e92bdaa58d584709011ee619bf5d699425"} err="failed to get container status \"6c1f37bbb573e63a908aea9d3b4362e92bdaa58d584709011ee619bf5d699425\": rpc error: code = NotFound desc = could not find container \"6c1f37bbb573e63a908aea9d3b4362e92bdaa58d584709011ee619bf5d699425\": container with ID starting with 6c1f37bbb573e63a908aea9d3b4362e92bdaa58d584709011ee619bf5d699425 not found: ID does not exist"
Sep 19 11:10:12.474130 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.474108    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"]
Sep 19 11:10:12.477484 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:12.477461    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-9b3bef-84c9df8fb5-c6stz"]
Sep 19 11:10:13.002305 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:13.002275    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0933555e-5813-45a1-bbbd-0bd17362423b" path="/var/lib/kubelet/pods/0933555e-5813-45a1-bbbd-0bd17362423b/volumes"
Sep 19 11:10:18.428709 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:18.428679    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"]
Sep 19 11:10:18.429128 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:18.428887    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd" podUID="5915405f-7676-421d-9230-b3c30b27a115" containerName="istio-proxy" containerID="cri-o://feb837d77fce3cc330894319cecd49d7385d12150241403209380277fe375a57" gracePeriod=30
Sep 19 11:10:21.582608 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:21.582574    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7599657744-dccbf"]
Sep 19 11:10:21.583058 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:21.582747    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7599657744-dccbf" podUID="06077445-6116-47dc-b23d-ddb2e73db131" containerName="manager" containerID="cri-o://b6ade56169226ab6c92f778f1f8f257bcb82dbbf97c1eb8cf2e97d996424a99c" gracePeriod=30
Sep 19 11:10:21.832856 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:21.832803    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7599657744-dccbf"
Sep 19 11:10:21.898378 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:21.898347    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5frs\" (UniqueName: \"kubernetes.io/projected/06077445-6116-47dc-b23d-ddb2e73db131-kube-api-access-n5frs\") pod \"06077445-6116-47dc-b23d-ddb2e73db131\" (UID: \"06077445-6116-47dc-b23d-ddb2e73db131\") "
Sep 19 11:10:21.898500 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:21.898414    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06077445-6116-47dc-b23d-ddb2e73db131-webhook-cert\") pod \"06077445-6116-47dc-b23d-ddb2e73db131\" (UID: \"06077445-6116-47dc-b23d-ddb2e73db131\") "
Sep 19 11:10:21.898551 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:21.898506    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06077445-6116-47dc-b23d-ddb2e73db131-metrics-certs\") pod \"06077445-6116-47dc-b23d-ddb2e73db131\" (UID: \"06077445-6116-47dc-b23d-ddb2e73db131\") "
Sep 19 11:10:21.900534 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:21.900504    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06077445-6116-47dc-b23d-ddb2e73db131-kube-api-access-n5frs" (OuterVolumeSpecName: "kube-api-access-n5frs") pod "06077445-6116-47dc-b23d-ddb2e73db131" (UID: "06077445-6116-47dc-b23d-ddb2e73db131"). InnerVolumeSpecName "kube-api-access-n5frs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:10:21.900650 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:21.900623    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06077445-6116-47dc-b23d-ddb2e73db131-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "06077445-6116-47dc-b23d-ddb2e73db131" (UID: "06077445-6116-47dc-b23d-ddb2e73db131"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:10:21.900873 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:21.900857    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06077445-6116-47dc-b23d-ddb2e73db131-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "06077445-6116-47dc-b23d-ddb2e73db131" (UID: "06077445-6116-47dc-b23d-ddb2e73db131"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:10:21.999906 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:21.999880    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06077445-6116-47dc-b23d-ddb2e73db131-metrics-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:21.999906 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:21.999905    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5frs\" (UniqueName: \"kubernetes.io/projected/06077445-6116-47dc-b23d-ddb2e73db131-kube-api-access-n5frs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:22.000062 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:21.999914    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06077445-6116-47dc-b23d-ddb2e73db131-webhook-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:22.507560 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:22.507528    2576 generic.go:356] "Generic (PLEG): container finished" podID="06077445-6116-47dc-b23d-ddb2e73db131" containerID="b6ade56169226ab6c92f778f1f8f257bcb82dbbf97c1eb8cf2e97d996424a99c" exitCode=0
Sep 19 11:10:22.507732 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:22.507577    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7599657744-dccbf"
Sep 19 11:10:22.507732 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:22.507616    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7599657744-dccbf" event={"ID":"06077445-6116-47dc-b23d-ddb2e73db131","Type":"ContainerDied","Data":"b6ade56169226ab6c92f778f1f8f257bcb82dbbf97c1eb8cf2e97d996424a99c"}
Sep 19 11:10:22.507732 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:22.507680    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7599657744-dccbf" event={"ID":"06077445-6116-47dc-b23d-ddb2e73db131","Type":"ContainerDied","Data":"4030db093a0cef1743f4f51c4add2154893ab6c57d6b0cfc2af8a39d85e3ebac"}
Sep 19 11:10:22.507732 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:22.507705    2576 scope.go:117] "RemoveContainer" containerID="b6ade56169226ab6c92f778f1f8f257bcb82dbbf97c1eb8cf2e97d996424a99c"
Sep 19 11:10:22.517548 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:22.517533    2576 scope.go:117] "RemoveContainer" containerID="b6ade56169226ab6c92f778f1f8f257bcb82dbbf97c1eb8cf2e97d996424a99c"
Sep 19 11:10:22.517884 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:10:22.517861    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6ade56169226ab6c92f778f1f8f257bcb82dbbf97c1eb8cf2e97d996424a99c\": container with ID starting with b6ade56169226ab6c92f778f1f8f257bcb82dbbf97c1eb8cf2e97d996424a99c not found: ID does not exist" containerID="b6ade56169226ab6c92f778f1f8f257bcb82dbbf97c1eb8cf2e97d996424a99c"
Sep 19 11:10:22.517954 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:22.517890    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6ade56169226ab6c92f778f1f8f257bcb82dbbf97c1eb8cf2e97d996424a99c"} err="failed to get container status \"b6ade56169226ab6c92f778f1f8f257bcb82dbbf97c1eb8cf2e97d996424a99c\": rpc error: code = NotFound desc = could not find container \"b6ade56169226ab6c92f778f1f8f257bcb82dbbf97c1eb8cf2e97d996424a99c\": container with ID starting with b6ade56169226ab6c92f778f1f8f257bcb82dbbf97c1eb8cf2e97d996424a99c not found: ID does not exist"
Sep 19 11:10:22.528692 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:22.528673    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7599657744-dccbf"]
Sep 19 11:10:22.534351 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:22.534328    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7599657744-dccbf"]
Sep 19 11:10:23.001696 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.001653    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06077445-6116-47dc-b23d-ddb2e73db131" path="/var/lib/kubelet/pods/06077445-6116-47dc-b23d-ddb2e73db131/volumes"
Sep 19 11:10:23.685458 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.685439    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 11:10:23.813892 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.813836    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-credential-socket\") pod \"5915405f-7676-421d-9230-b3c30b27a115\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") "
Sep 19 11:10:23.813892 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.813869    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5915405f-7676-421d-9230-b3c30b27a115-istio-podinfo\") pod \"5915405f-7676-421d-9230-b3c30b27a115\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") "
Sep 19 11:10:23.813892 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.813889    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5915405f-7676-421d-9230-b3c30b27a115-istio-token\") pod \"5915405f-7676-421d-9230-b3c30b27a115\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") "
Sep 19 11:10:23.814162 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.813910    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5915405f-7676-421d-9230-b3c30b27a115-istiod-ca-cert\") pod \"5915405f-7676-421d-9230-b3c30b27a115\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") "
Sep 19 11:10:23.814162 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.813941    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-istio-data\") pod \"5915405f-7676-421d-9230-b3c30b27a115\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") "
Sep 19 11:10:23.814162 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.813982    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-workload-certs\") pod \"5915405f-7676-421d-9230-b3c30b27a115\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") "
Sep 19 11:10:23.814162 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.814000    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-workload-socket\") pod \"5915405f-7676-421d-9230-b3c30b27a115\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") "
Sep 19 11:10:23.814162 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.814026    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-istio-envoy\") pod \"5915405f-7676-421d-9230-b3c30b27a115\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") "
Sep 19 11:10:23.814162 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.814064    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpz6b\" (UniqueName: \"kubernetes.io/projected/5915405f-7676-421d-9230-b3c30b27a115-kube-api-access-qpz6b\") pod \"5915405f-7676-421d-9230-b3c30b27a115\" (UID: \"5915405f-7676-421d-9230-b3c30b27a115\") "
Sep 19 11:10:23.814162 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.814120    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5915405f-7676-421d-9230-b3c30b27a115" (UID: "5915405f-7676-421d-9230-b3c30b27a115"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:10:23.814535 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.814319    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5915405f-7676-421d-9230-b3c30b27a115" (UID: "5915405f-7676-421d-9230-b3c30b27a115"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:10:23.814535 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.814334    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5915405f-7676-421d-9230-b3c30b27a115" (UID: "5915405f-7676-421d-9230-b3c30b27a115"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:10:23.814535 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.814350    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-istio-data" (OuterVolumeSpecName: "istio-data") pod "5915405f-7676-421d-9230-b3c30b27a115" (UID: "5915405f-7676-421d-9230-b3c30b27a115"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:10:23.814535 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.814385    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:23.814535 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.814390    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5915405f-7676-421d-9230-b3c30b27a115-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5915405f-7676-421d-9230-b3c30b27a115" (UID: "5915405f-7676-421d-9230-b3c30b27a115"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:10:23.816383 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.816362    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5915405f-7676-421d-9230-b3c30b27a115-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5915405f-7676-421d-9230-b3c30b27a115" (UID: "5915405f-7676-421d-9230-b3c30b27a115"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 11:10:23.816383 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.816357    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5915405f-7676-421d-9230-b3c30b27a115" (UID: "5915405f-7676-421d-9230-b3c30b27a115"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:10:23.816514 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.816389    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5915405f-7676-421d-9230-b3c30b27a115-istio-token" (OuterVolumeSpecName: "istio-token") pod "5915405f-7676-421d-9230-b3c30b27a115" (UID: "5915405f-7676-421d-9230-b3c30b27a115"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:10:23.816567 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.816549    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5915405f-7676-421d-9230-b3c30b27a115-kube-api-access-qpz6b" (OuterVolumeSpecName: "kube-api-access-qpz6b") pod "5915405f-7676-421d-9230-b3c30b27a115" (UID: "5915405f-7676-421d-9230-b3c30b27a115"). InnerVolumeSpecName "kube-api-access-qpz6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:10:23.915276 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.915247    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5915405f-7676-421d-9230-b3c30b27a115-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:23.915276 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.915273    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5915405f-7676-421d-9230-b3c30b27a115-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:23.915428 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.915287    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5915405f-7676-421d-9230-b3c30b27a115-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:23.915428 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.915298    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:23.915428 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.915309    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:23.915428 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.915321    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:23.915428 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.915333    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5915405f-7676-421d-9230-b3c30b27a115-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:23.915428 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:23.915346    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpz6b\" (UniqueName: \"kubernetes.io/projected/5915405f-7676-421d-9230-b3c30b27a115-kube-api-access-qpz6b\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:10:24.520193 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:24.520157    2576 generic.go:356] "Generic (PLEG): container finished" podID="5915405f-7676-421d-9230-b3c30b27a115" containerID="feb837d77fce3cc330894319cecd49d7385d12150241403209380277fe375a57" exitCode=0
Sep 19 11:10:24.520663 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:24.520207    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"
Sep 19 11:10:24.520663 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:24.520208    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd" event={"ID":"5915405f-7676-421d-9230-b3c30b27a115","Type":"ContainerDied","Data":"feb837d77fce3cc330894319cecd49d7385d12150241403209380277fe375a57"}
Sep 19 11:10:24.520663 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:24.520323    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd" event={"ID":"5915405f-7676-421d-9230-b3c30b27a115","Type":"ContainerDied","Data":"59fed62dfa2fb2256087e27d5b5a5253007f73ad6487b993f8fa6e41661e4c9c"}
Sep 19 11:10:24.520663 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:24.520352    2576 scope.go:117] "RemoveContainer" containerID="feb837d77fce3cc330894319cecd49d7385d12150241403209380277fe375a57"
Sep 19 11:10:24.532709 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:24.532686    2576 scope.go:117] "RemoveContainer" containerID="201b6f91475dbaf0514b38f1f4bf6f5b53552f9ea01cbf117c00c5742c63c6a4"
Sep 19 11:10:24.541941 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:24.541920    2576 scope.go:117] "RemoveContainer" containerID="feb837d77fce3cc330894319cecd49d7385d12150241403209380277fe375a57"
Sep 19 11:10:24.542244 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:10:24.542223    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feb837d77fce3cc330894319cecd49d7385d12150241403209380277fe375a57\": container with ID starting with feb837d77fce3cc330894319cecd49d7385d12150241403209380277fe375a57 not found: ID does not exist" containerID="feb837d77fce3cc330894319cecd49d7385d12150241403209380277fe375a57"
Sep 19 11:10:24.542324 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:24.542256    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feb837d77fce3cc330894319cecd49d7385d12150241403209380277fe375a57"} err="failed to get container status \"feb837d77fce3cc330894319cecd49d7385d12150241403209380277fe375a57\": rpc error: code = NotFound desc = could not find container \"feb837d77fce3cc330894319cecd49d7385d12150241403209380277fe375a57\": container with ID starting with feb837d77fce3cc330894319cecd49d7385d12150241403209380277fe375a57 not found: ID does not exist"
Sep 19 11:10:24.542324 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:24.542279    2576 scope.go:117] "RemoveContainer" containerID="201b6f91475dbaf0514b38f1f4bf6f5b53552f9ea01cbf117c00c5742c63c6a4"
Sep 19 11:10:24.542570 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:10:24.542553    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"201b6f91475dbaf0514b38f1f4bf6f5b53552f9ea01cbf117c00c5742c63c6a4\": container with ID starting with 201b6f91475dbaf0514b38f1f4bf6f5b53552f9ea01cbf117c00c5742c63c6a4 not found: ID does not exist" containerID="201b6f91475dbaf0514b38f1f4bf6f5b53552f9ea01cbf117c00c5742c63c6a4"
Sep 19 11:10:24.542614 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:24.542576    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"201b6f91475dbaf0514b38f1f4bf6f5b53552f9ea01cbf117c00c5742c63c6a4"} err="failed to get container status \"201b6f91475dbaf0514b38f1f4bf6f5b53552f9ea01cbf117c00c5742c63c6a4\": rpc error: code = NotFound desc = could not find container \"201b6f91475dbaf0514b38f1f4bf6f5b53552f9ea01cbf117c00c5742c63c6a4\": container with ID starting with 201b6f91475dbaf0514b38f1f4bf6f5b53552f9ea01cbf117c00c5742c63c6a4 not found: ID does not exist"
Sep 19 11:10:24.542705 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:24.542688    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"]
Sep 19 11:10:24.547926 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:24.547905    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9b3bef-openshift-default-556565dcd4-r7jnd"]
Sep 19 11:10:25.008813 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:10:25.008778    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5915405f-7676-421d-9230-b3c30b27a115" path="/var/lib/kubelet/pods/5915405f-7676-421d-9230-b3c30b27a115/volumes"
Sep 19 11:11:23.329012 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.328978    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7599657744-qp8hp"]
Sep 19 11:11:23.329498 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329375    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81a466e0-48b3-4452-b24b-b7b5cf25ee13" containerName="maas-api"
Sep 19 11:11:23.329498 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329391    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="81a466e0-48b3-4452-b24b-b7b5cf25ee13" containerName="maas-api"
Sep 19 11:11:23.329498 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329413    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3142ef93-54d6-463d-985f-6fb7b28eadfe" containerName="istio-proxy"
Sep 19 11:11:23.329498 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329419    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3142ef93-54d6-463d-985f-6fb7b28eadfe" containerName="istio-proxy"
Sep 19 11:11:23.329498 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329425    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0933555e-5813-45a1-bbbd-0bd17362423b" containerName="maas-api"
Sep 19 11:11:23.329498 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329432    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0933555e-5813-45a1-bbbd-0bd17362423b" containerName="maas-api"
Sep 19 11:11:23.329498 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329443    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3142ef93-54d6-463d-985f-6fb7b28eadfe" containerName="istio-proxy"
Sep 19 11:11:23.329498 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329449    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3142ef93-54d6-463d-985f-6fb7b28eadfe" containerName="istio-proxy"
Sep 19 11:11:23.329498 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329458    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06077445-6116-47dc-b23d-ddb2e73db131" containerName="manager"
Sep 19 11:11:23.329498 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329463    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="06077445-6116-47dc-b23d-ddb2e73db131" containerName="manager"
Sep 19 11:11:23.329498 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329470    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95e41548-0b16-4415-aa2d-922806b657c5" containerName="istio-proxy"
Sep 19 11:11:23.329498 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329475    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="95e41548-0b16-4415-aa2d-922806b657c5" containerName="istio-proxy"
Sep 19 11:11:23.329498 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329482    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc608e31-f041-4f90-ae28-8e8932c9b2ca" containerName="maas-api"
Sep 19 11:11:23.329498 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329487    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc608e31-f041-4f90-ae28-8e8932c9b2ca" containerName="maas-api"
Sep 19 11:11:23.329498 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329492    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26a85e00-629b-4d45-8984-377f1e13123b" containerName="maas-api"
Sep 19 11:11:23.329498 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329496    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="26a85e00-629b-4d45-8984-377f1e13123b" containerName="maas-api"
Sep 19 11:11:23.329498 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329504    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95e41548-0b16-4415-aa2d-922806b657c5" containerName="istio-proxy"
Sep 19 11:11:23.330099 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329510    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="95e41548-0b16-4415-aa2d-922806b657c5" containerName="istio-proxy"
Sep 19 11:11:23.330099 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329517    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5915405f-7676-421d-9230-b3c30b27a115" containerName="istio-proxy"
Sep 19 11:11:23.330099 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329522    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5915405f-7676-421d-9230-b3c30b27a115" containerName="istio-proxy"
Sep 19 11:11:23.330099 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329527    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5915405f-7676-421d-9230-b3c30b27a115" containerName="istio-proxy"
Sep 19 11:11:23.330099 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329532    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5915405f-7676-421d-9230-b3c30b27a115" containerName="istio-proxy"
Sep 19 11:11:23.330099 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329585    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0933555e-5813-45a1-bbbd-0bd17362423b" containerName="maas-api"
Sep 19 11:11:23.330099 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329594    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="81a466e0-48b3-4452-b24b-b7b5cf25ee13" containerName="maas-api"
Sep 19 11:11:23.330099 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329600    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3142ef93-54d6-463d-985f-6fb7b28eadfe" containerName="istio-proxy"
Sep 19 11:11:23.330099 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329606    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc608e31-f041-4f90-ae28-8e8932c9b2ca" containerName="maas-api"
Sep 19 11:11:23.330099 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329613    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="26a85e00-629b-4d45-8984-377f1e13123b" containerName="maas-api"
Sep 19 11:11:23.330099 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329619    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5915405f-7676-421d-9230-b3c30b27a115" containerName="istio-proxy"
Sep 19 11:11:23.330099 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329625    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5915405f-7676-421d-9230-b3c30b27a115" containerName="istio-proxy"
Sep 19 11:11:23.330099 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329631    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="95e41548-0b16-4415-aa2d-922806b657c5" containerName="istio-proxy"
Sep 19 11:11:23.330099 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329637    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="95e41548-0b16-4415-aa2d-922806b657c5" containerName="istio-proxy"
Sep 19 11:11:23.330099 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.329643    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="06077445-6116-47dc-b23d-ddb2e73db131" containerName="manager"
Sep 19 11:11:23.331727 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.331710    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7599657744-qp8hp"
Sep 19 11:11:23.335079 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.335054    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6lwg4\""
Sep 19 11:11:23.335323 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.335304    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 19 11:11:23.335458 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.335440    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 19 11:11:23.335511 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.335445    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 19 11:11:23.343705 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.343682    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7599657744-qp8hp"]
Sep 19 11:11:23.392107 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.392078    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/517d1f6f-9861-4e92-963b-97c018ecad44-metrics-certs\") pod \"maas-controller-7599657744-qp8hp\" (UID: \"517d1f6f-9861-4e92-963b-97c018ecad44\") " pod="opendatahub/maas-controller-7599657744-qp8hp"
Sep 19 11:11:23.392282 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.392134    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/517d1f6f-9861-4e92-963b-97c018ecad44-webhook-cert\") pod \"maas-controller-7599657744-qp8hp\" (UID: \"517d1f6f-9861-4e92-963b-97c018ecad44\") " pod="opendatahub/maas-controller-7599657744-qp8hp"
Sep 19 11:11:23.392282 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.392202    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmdpk\" (UniqueName: \"kubernetes.io/projected/517d1f6f-9861-4e92-963b-97c018ecad44-kube-api-access-zmdpk\") pod \"maas-controller-7599657744-qp8hp\" (UID: \"517d1f6f-9861-4e92-963b-97c018ecad44\") " pod="opendatahub/maas-controller-7599657744-qp8hp"
Sep 19 11:11:23.492855 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.492819    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/517d1f6f-9861-4e92-963b-97c018ecad44-metrics-certs\") pod \"maas-controller-7599657744-qp8hp\" (UID: \"517d1f6f-9861-4e92-963b-97c018ecad44\") " pod="opendatahub/maas-controller-7599657744-qp8hp"
Sep 19 11:11:23.493037 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.492865    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/517d1f6f-9861-4e92-963b-97c018ecad44-webhook-cert\") pod \"maas-controller-7599657744-qp8hp\" (UID: \"517d1f6f-9861-4e92-963b-97c018ecad44\") " pod="opendatahub/maas-controller-7599657744-qp8hp"
Sep 19 11:11:23.493037 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.492898    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmdpk\" (UniqueName: \"kubernetes.io/projected/517d1f6f-9861-4e92-963b-97c018ecad44-kube-api-access-zmdpk\") pod \"maas-controller-7599657744-qp8hp\" (UID: \"517d1f6f-9861-4e92-963b-97c018ecad44\") " pod="opendatahub/maas-controller-7599657744-qp8hp"
Sep 19 11:11:23.495426 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.495391    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/517d1f6f-9861-4e92-963b-97c018ecad44-metrics-certs\") pod \"maas-controller-7599657744-qp8hp\" (UID: \"517d1f6f-9861-4e92-963b-97c018ecad44\") " pod="opendatahub/maas-controller-7599657744-qp8hp"
Sep 19 11:11:23.495531 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.495462    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/517d1f6f-9861-4e92-963b-97c018ecad44-webhook-cert\") pod \"maas-controller-7599657744-qp8hp\" (UID: \"517d1f6f-9861-4e92-963b-97c018ecad44\") " pod="opendatahub/maas-controller-7599657744-qp8hp"
Sep 19 11:11:23.500681 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.500657    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmdpk\" (UniqueName: \"kubernetes.io/projected/517d1f6f-9861-4e92-963b-97c018ecad44-kube-api-access-zmdpk\") pod \"maas-controller-7599657744-qp8hp\" (UID: \"517d1f6f-9861-4e92-963b-97c018ecad44\") " pod="opendatahub/maas-controller-7599657744-qp8hp"
Sep 19 11:11:23.643250 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.643210    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7599657744-qp8hp"
Sep 19 11:11:23.914849 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.914824    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7599657744-qp8hp"]
Sep 19 11:11:23.917408 ip-10-0-142-165 kubenswrapper[2576]: W0919 11:11:23.917381    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod517d1f6f_9861_4e92_963b_97c018ecad44.slice/crio-54388ad35a28a05f538360d03a83d482df703d7dbd0818f7af3e24374846b423 WatchSource:0}: Error finding container 54388ad35a28a05f538360d03a83d482df703d7dbd0818f7af3e24374846b423: Status 404 returned error can't find the container with id 54388ad35a28a05f538360d03a83d482df703d7dbd0818f7af3e24374846b423
Sep 19 11:11:23.919164 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:23.919148    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 11:11:24.833957 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:24.833859    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7599657744-qp8hp" event={"ID":"517d1f6f-9861-4e92-963b-97c018ecad44","Type":"ContainerStarted","Data":"421087c556ef59c4a7a868dc84297fbfb0425de35166e2876fd7588d1d74ecbd"}
Sep 19 11:11:24.833957 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:24.833924    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7599657744-qp8hp" event={"ID":"517d1f6f-9861-4e92-963b-97c018ecad44","Type":"ContainerStarted","Data":"54388ad35a28a05f538360d03a83d482df703d7dbd0818f7af3e24374846b423"}
Sep 19 11:11:24.834425 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:24.833983    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7599657744-qp8hp"
Sep 19 11:11:24.850725 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:24.850670    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7599657744-qp8hp" podStartSLOduration=1.3047771670000001 podStartE2EDuration="1.850650162s" totalImagesPullingTime="545.872995ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:11:24.849357669 +0000 UTC m=+2094.345663772" watchObservedRunningTime="2026-09-19 11:11:24.850650162 +0000 UTC m=+2094.346956256"
Sep 19 11:11:31.114146 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:31.114121    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 11:11:31.121379 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:31.121352    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 11:11:35.841364 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:35.841333    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7599657744-qp8hp"
Sep 19 11:11:45.197745 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.197713    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7599657744-qp8hp"]
Sep 19 11:11:45.198425 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.197923    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7599657744-qp8hp" podUID="517d1f6f-9861-4e92-963b-97c018ecad44" containerName="manager" containerID="cri-o://421087c556ef59c4a7a868dc84297fbfb0425de35166e2876fd7588d1d74ecbd" gracePeriod=30
Sep 19 11:11:45.441508 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.441486    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7599657744-qp8hp"
Sep 19 11:11:45.483086 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.483007    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmdpk\" (UniqueName: \"kubernetes.io/projected/517d1f6f-9861-4e92-963b-97c018ecad44-kube-api-access-zmdpk\") pod \"517d1f6f-9861-4e92-963b-97c018ecad44\" (UID: \"517d1f6f-9861-4e92-963b-97c018ecad44\") "
Sep 19 11:11:45.483206 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.483161    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/517d1f6f-9861-4e92-963b-97c018ecad44-metrics-certs\") pod \"517d1f6f-9861-4e92-963b-97c018ecad44\" (UID: \"517d1f6f-9861-4e92-963b-97c018ecad44\") "
Sep 19 11:11:45.483206 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.483199    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/517d1f6f-9861-4e92-963b-97c018ecad44-webhook-cert\") pod \"517d1f6f-9861-4e92-963b-97c018ecad44\" (UID: \"517d1f6f-9861-4e92-963b-97c018ecad44\") "
Sep 19 11:11:45.485318 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.485289    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/517d1f6f-9861-4e92-963b-97c018ecad44-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "517d1f6f-9861-4e92-963b-97c018ecad44" (UID: "517d1f6f-9861-4e92-963b-97c018ecad44"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:11:45.485436 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.485350    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/517d1f6f-9861-4e92-963b-97c018ecad44-kube-api-access-zmdpk" (OuterVolumeSpecName: "kube-api-access-zmdpk") pod "517d1f6f-9861-4e92-963b-97c018ecad44" (UID: "517d1f6f-9861-4e92-963b-97c018ecad44"). InnerVolumeSpecName "kube-api-access-zmdpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:11:45.485436 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.485410    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/517d1f6f-9861-4e92-963b-97c018ecad44-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "517d1f6f-9861-4e92-963b-97c018ecad44" (UID: "517d1f6f-9861-4e92-963b-97c018ecad44"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:11:45.584328 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.584293    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmdpk\" (UniqueName: \"kubernetes.io/projected/517d1f6f-9861-4e92-963b-97c018ecad44-kube-api-access-zmdpk\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:11:45.584328 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.584318    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/517d1f6f-9861-4e92-963b-97c018ecad44-metrics-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:11:45.584328 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.584327    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/517d1f6f-9861-4e92-963b-97c018ecad44-webhook-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:11:45.944180 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.944144    2576 generic.go:356] "Generic (PLEG): container finished" podID="517d1f6f-9861-4e92-963b-97c018ecad44" containerID="421087c556ef59c4a7a868dc84297fbfb0425de35166e2876fd7588d1d74ecbd" exitCode=0
Sep 19 11:11:45.944328 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.944189    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7599657744-qp8hp"
Sep 19 11:11:45.944328 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.944231    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7599657744-qp8hp" event={"ID":"517d1f6f-9861-4e92-963b-97c018ecad44","Type":"ContainerDied","Data":"421087c556ef59c4a7a868dc84297fbfb0425de35166e2876fd7588d1d74ecbd"}
Sep 19 11:11:45.944328 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.944281    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7599657744-qp8hp" event={"ID":"517d1f6f-9861-4e92-963b-97c018ecad44","Type":"ContainerDied","Data":"54388ad35a28a05f538360d03a83d482df703d7dbd0818f7af3e24374846b423"}
Sep 19 11:11:45.944328 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.944302    2576 scope.go:117] "RemoveContainer" containerID="421087c556ef59c4a7a868dc84297fbfb0425de35166e2876fd7588d1d74ecbd"
Sep 19 11:11:45.953734 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.953717    2576 scope.go:117] "RemoveContainer" containerID="421087c556ef59c4a7a868dc84297fbfb0425de35166e2876fd7588d1d74ecbd"
Sep 19 11:11:45.954004 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:11:45.953987    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"421087c556ef59c4a7a868dc84297fbfb0425de35166e2876fd7588d1d74ecbd\": container with ID starting with 421087c556ef59c4a7a868dc84297fbfb0425de35166e2876fd7588d1d74ecbd not found: ID does not exist" containerID="421087c556ef59c4a7a868dc84297fbfb0425de35166e2876fd7588d1d74ecbd"
Sep 19 11:11:45.954120 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.954012    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"421087c556ef59c4a7a868dc84297fbfb0425de35166e2876fd7588d1d74ecbd"} err="failed to get container status \"421087c556ef59c4a7a868dc84297fbfb0425de35166e2876fd7588d1d74ecbd\": rpc error: code = NotFound desc = could not find container \"421087c556ef59c4a7a868dc84297fbfb0425de35166e2876fd7588d1d74ecbd\": container with ID starting with 421087c556ef59c4a7a868dc84297fbfb0425de35166e2876fd7588d1d74ecbd not found: ID does not exist"
Sep 19 11:11:45.965325 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.965298    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7599657744-qp8hp"]
Sep 19 11:11:45.968735 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:45.968710    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7599657744-qp8hp"]
Sep 19 11:11:46.776483 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.776446    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7599657744-8c88j"]
Sep 19 11:11:46.776860 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.776834    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="517d1f6f-9861-4e92-963b-97c018ecad44" containerName="manager"
Sep 19 11:11:46.776860 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.776844    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="517d1f6f-9861-4e92-963b-97c018ecad44" containerName="manager"
Sep 19 11:11:46.776937 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.776899    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="517d1f6f-9861-4e92-963b-97c018ecad44" containerName="manager"
Sep 19 11:11:46.776937 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.776910    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3142ef93-54d6-463d-985f-6fb7b28eadfe" containerName="istio-proxy"
Sep 19 11:11:46.781257 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.781241    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7599657744-8c88j"
Sep 19 11:11:46.784634 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.784603    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 19 11:11:46.784790 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.784616    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6lwg4\""
Sep 19 11:11:46.784790 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.784679    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 19 11:11:46.784790 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.784682    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 19 11:11:46.788559 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.788536    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7599657744-8c88j"]
Sep 19 11:11:46.792259 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.792233    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77ce7462-639a-46b0-b250-33ee0966ae6b-webhook-cert\") pod \"maas-controller-7599657744-8c88j\" (UID: \"77ce7462-639a-46b0-b250-33ee0966ae6b\") " pod="opendatahub/maas-controller-7599657744-8c88j"
Sep 19 11:11:46.792378 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.792288    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77ce7462-639a-46b0-b250-33ee0966ae6b-metrics-certs\") pod \"maas-controller-7599657744-8c88j\" (UID: \"77ce7462-639a-46b0-b250-33ee0966ae6b\") " pod="opendatahub/maas-controller-7599657744-8c88j"
Sep 19 11:11:46.792448 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.792380    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zttsp\" (UniqueName: \"kubernetes.io/projected/77ce7462-639a-46b0-b250-33ee0966ae6b-kube-api-access-zttsp\") pod \"maas-controller-7599657744-8c88j\" (UID: \"77ce7462-639a-46b0-b250-33ee0966ae6b\") " pod="opendatahub/maas-controller-7599657744-8c88j"
Sep 19 11:11:46.893490 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.893463    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77ce7462-639a-46b0-b250-33ee0966ae6b-metrics-certs\") pod \"maas-controller-7599657744-8c88j\" (UID: \"77ce7462-639a-46b0-b250-33ee0966ae6b\") " pod="opendatahub/maas-controller-7599657744-8c88j"
Sep 19 11:11:46.893650 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.893503    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zttsp\" (UniqueName: \"kubernetes.io/projected/77ce7462-639a-46b0-b250-33ee0966ae6b-kube-api-access-zttsp\") pod \"maas-controller-7599657744-8c88j\" (UID: \"77ce7462-639a-46b0-b250-33ee0966ae6b\") " pod="opendatahub/maas-controller-7599657744-8c88j"
Sep 19 11:11:46.893650 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.893575    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77ce7462-639a-46b0-b250-33ee0966ae6b-webhook-cert\") pod \"maas-controller-7599657744-8c88j\" (UID: \"77ce7462-639a-46b0-b250-33ee0966ae6b\") " pod="opendatahub/maas-controller-7599657744-8c88j"
Sep 19 11:11:46.896061 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.896020    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77ce7462-639a-46b0-b250-33ee0966ae6b-webhook-cert\") pod \"maas-controller-7599657744-8c88j\" (UID: \"77ce7462-639a-46b0-b250-33ee0966ae6b\") " pod="opendatahub/maas-controller-7599657744-8c88j"
Sep 19 11:11:46.896178 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.896076    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77ce7462-639a-46b0-b250-33ee0966ae6b-metrics-certs\") pod \"maas-controller-7599657744-8c88j\" (UID: \"77ce7462-639a-46b0-b250-33ee0966ae6b\") " pod="opendatahub/maas-controller-7599657744-8c88j"
Sep 19 11:11:46.901460 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:46.901440    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zttsp\" (UniqueName: \"kubernetes.io/projected/77ce7462-639a-46b0-b250-33ee0966ae6b-kube-api-access-zttsp\") pod \"maas-controller-7599657744-8c88j\" (UID: \"77ce7462-639a-46b0-b250-33ee0966ae6b\") " pod="opendatahub/maas-controller-7599657744-8c88j"
Sep 19 11:11:47.001022 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:47.000991    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="517d1f6f-9861-4e92-963b-97c018ecad44" path="/var/lib/kubelet/pods/517d1f6f-9861-4e92-963b-97c018ecad44/volumes"
Sep 19 11:11:47.093252 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:47.093224    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7599657744-8c88j"
Sep 19 11:11:47.365839 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:47.365812    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7599657744-8c88j"]
Sep 19 11:11:47.368082 ip-10-0-142-165 kubenswrapper[2576]: W0919 11:11:47.368027    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77ce7462_639a_46b0_b250_33ee0966ae6b.slice/crio-607aeb19c423a89545c46d55e0b4ab6dcbe670e6d016cbfc0ec5a5552e568469 WatchSource:0}: Error finding container 607aeb19c423a89545c46d55e0b4ab6dcbe670e6d016cbfc0ec5a5552e568469: Status 404 returned error can't find the container with id 607aeb19c423a89545c46d55e0b4ab6dcbe670e6d016cbfc0ec5a5552e568469
Sep 19 11:11:47.959187 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:47.959089    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7599657744-8c88j" event={"ID":"77ce7462-639a-46b0-b250-33ee0966ae6b","Type":"ContainerStarted","Data":"bc37e9e6c65059e25f5b3ba3463ad901b575be60a12b1b3a4064fc94da9a3bdd"}
Sep 19 11:11:47.959545 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:47.959196    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7599657744-8c88j" event={"ID":"77ce7462-639a-46b0-b250-33ee0966ae6b","Type":"ContainerStarted","Data":"607aeb19c423a89545c46d55e0b4ab6dcbe670e6d016cbfc0ec5a5552e568469"}
Sep 19 11:11:47.959545 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:47.959227    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7599657744-8c88j"
Sep 19 11:11:47.975765 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:47.975714    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7599657744-8c88j" podStartSLOduration=1.655603114 podStartE2EDuration="1.975697373s" totalImagesPullingTime="320.094259ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:11:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:11:47.974490994 +0000 UTC m=+2117.470797085" watchObservedRunningTime="2026-09-19 11:11:47.975697373 +0000 UTC m=+2117.472003464"
Sep 19 11:11:58.967790 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:11:58.967756    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7599657744-8c88j"
Sep 19 11:15:41.426368 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:41.426324    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk"]
Sep 19 11:15:41.430103 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:41.430076    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk"
Sep 19 11:15:41.440579 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:41.440539    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk"]
Sep 19 11:15:41.509334 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:41.509296    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7217fcef-611d-47b6-ba5d-e467719042cf-metrics-certs\") pod \"maas-api-6d474ff8d6-8skgk\" (UID: \"7217fcef-611d-47b6-ba5d-e467719042cf\") " pod="odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk"
Sep 19 11:15:41.509545 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:41.509390    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwcmz\" (UniqueName: \"kubernetes.io/projected/7217fcef-611d-47b6-ba5d-e467719042cf-kube-api-access-kwcmz\") pod \"maas-api-6d474ff8d6-8skgk\" (UID: \"7217fcef-611d-47b6-ba5d-e467719042cf\") " pod="odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk"
Sep 19 11:15:41.509545 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:41.509414    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7217fcef-611d-47b6-ba5d-e467719042cf-openshift-service-ca\") pod \"maas-api-6d474ff8d6-8skgk\" (UID: \"7217fcef-611d-47b6-ba5d-e467719042cf\") " pod="odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk"
Sep 19 11:15:41.509643 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:41.509547    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7217fcef-611d-47b6-ba5d-e467719042cf-maas-api-tls\") pod \"maas-api-6d474ff8d6-8skgk\" (UID: \"7217fcef-611d-47b6-ba5d-e467719042cf\") " pod="odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk"
Sep 19 11:15:41.610550 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:41.610508    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7217fcef-611d-47b6-ba5d-e467719042cf-maas-api-tls\") pod \"maas-api-6d474ff8d6-8skgk\" (UID: \"7217fcef-611d-47b6-ba5d-e467719042cf\") " pod="odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk"
Sep 19 11:15:41.610550 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:41.610555    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7217fcef-611d-47b6-ba5d-e467719042cf-metrics-certs\") pod \"maas-api-6d474ff8d6-8skgk\" (UID: \"7217fcef-611d-47b6-ba5d-e467719042cf\") " pod="odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk"
Sep 19 11:15:41.610806 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:41.610600    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwcmz\" (UniqueName: \"kubernetes.io/projected/7217fcef-611d-47b6-ba5d-e467719042cf-kube-api-access-kwcmz\") pod \"maas-api-6d474ff8d6-8skgk\" (UID: \"7217fcef-611d-47b6-ba5d-e467719042cf\") " pod="odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk"
Sep 19 11:15:41.610806 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:41.610620    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7217fcef-611d-47b6-ba5d-e467719042cf-openshift-service-ca\") pod \"maas-api-6d474ff8d6-8skgk\" (UID: \"7217fcef-611d-47b6-ba5d-e467719042cf\") " pod="odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk"
Sep 19 11:15:41.611390 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:41.611358    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7217fcef-611d-47b6-ba5d-e467719042cf-openshift-service-ca\") pod \"maas-api-6d474ff8d6-8skgk\" (UID: \"7217fcef-611d-47b6-ba5d-e467719042cf\") " pod="odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk"
Sep 19 11:15:41.613410 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:41.613388    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7217fcef-611d-47b6-ba5d-e467719042cf-metrics-certs\") pod \"maas-api-6d474ff8d6-8skgk\" (UID: \"7217fcef-611d-47b6-ba5d-e467719042cf\") " pod="odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk"
Sep 19 11:15:41.613494 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:41.613398    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7217fcef-611d-47b6-ba5d-e467719042cf-maas-api-tls\") pod \"maas-api-6d474ff8d6-8skgk\" (UID: \"7217fcef-611d-47b6-ba5d-e467719042cf\") " pod="odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk"
Sep 19 11:15:41.618252 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:41.618225    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwcmz\" (UniqueName: \"kubernetes.io/projected/7217fcef-611d-47b6-ba5d-e467719042cf-kube-api-access-kwcmz\") pod \"maas-api-6d474ff8d6-8skgk\" (UID: \"7217fcef-611d-47b6-ba5d-e467719042cf\") " pod="odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk"
Sep 19 11:15:41.744693 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:41.744588    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk"
Sep 19 11:15:42.020440 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:42.020411    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk"]
Sep 19 11:15:42.023135 ip-10-0-142-165 kubenswrapper[2576]: W0919 11:15:42.023099    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7217fcef_611d_47b6_ba5d_e467719042cf.slice/crio-f8055745c8eb625dd75822f07a2ef612edb8f08b3c9234a7c66ac416506d9e01 WatchSource:0}: Error finding container f8055745c8eb625dd75822f07a2ef612edb8f08b3c9234a7c66ac416506d9e01: Status 404 returned error can't find the container with id f8055745c8eb625dd75822f07a2ef612edb8f08b3c9234a7c66ac416506d9e01
Sep 19 11:15:42.192562 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:42.192525    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk" event={"ID":"7217fcef-611d-47b6-ba5d-e467719042cf","Type":"ContainerStarted","Data":"f8055745c8eb625dd75822f07a2ef612edb8f08b3c9234a7c66ac416506d9e01"}
Sep 19 11:15:43.199252 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:43.199212    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk" event={"ID":"7217fcef-611d-47b6-ba5d-e467719042cf","Type":"ContainerStarted","Data":"32ba82814e0d5470cef34bc15056eb8d0f979b786fe67dc2f3dea810ad8137d3"}
Sep 19 11:15:43.199675 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:43.199278    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk"
Sep 19 11:15:43.214862 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:43.214798    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk" podStartSLOduration=1.845506616 podStartE2EDuration="2.214776408s" totalImagesPullingTime="369.269792ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:15:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:15:43.214104974 +0000 UTC m=+2352.710411066" watchObservedRunningTime="2026-09-19 11:15:43.214776408 +0000 UTC m=+2352.711082500"
Sep 19 11:15:49.210130 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:49.210097    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d474ff8d6-8skgk"
Sep 19 11:15:49.253726 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:49.253683    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"]
Sep 19 11:15:49.253984 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:49.253958    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz" podUID="1982fcfa-c5e2-42b7-8cd4-4f08da150620" containerName="maas-api" containerID="cri-o://1dec620c6ef9bf8904072c064c06b3540db2ced662ff59d7469f2a484d94668b" gracePeriod=30
Sep 19 11:15:50.504597 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:50.504562    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"
Sep 19 11:15:50.601108 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:50.600977    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1982fcfa-c5e2-42b7-8cd4-4f08da150620-maas-api-tls\") pod \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\" (UID: \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\") "
Sep 19 11:15:50.601108 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:50.601077    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xd567\" (UniqueName: \"kubernetes.io/projected/1982fcfa-c5e2-42b7-8cd4-4f08da150620-kube-api-access-xd567\") pod \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\" (UID: \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\") "
Sep 19 11:15:50.601320 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:50.601128    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1982fcfa-c5e2-42b7-8cd4-4f08da150620-openshift-service-ca\") pod \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\" (UID: \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\") "
Sep 19 11:15:50.601320 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:50.601188    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1982fcfa-c5e2-42b7-8cd4-4f08da150620-metrics-certs\") pod \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\" (UID: \"1982fcfa-c5e2-42b7-8cd4-4f08da150620\") "
Sep 19 11:15:50.601579 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:50.601550    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1982fcfa-c5e2-42b7-8cd4-4f08da150620-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1982fcfa-c5e2-42b7-8cd4-4f08da150620" (UID: "1982fcfa-c5e2-42b7-8cd4-4f08da150620"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:15:50.603534 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:50.603487    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1982fcfa-c5e2-42b7-8cd4-4f08da150620-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1982fcfa-c5e2-42b7-8cd4-4f08da150620" (UID: "1982fcfa-c5e2-42b7-8cd4-4f08da150620"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:15:50.603534 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:50.603525    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1982fcfa-c5e2-42b7-8cd4-4f08da150620-kube-api-access-xd567" (OuterVolumeSpecName: "kube-api-access-xd567") pod "1982fcfa-c5e2-42b7-8cd4-4f08da150620" (UID: "1982fcfa-c5e2-42b7-8cd4-4f08da150620"). InnerVolumeSpecName "kube-api-access-xd567". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:15:50.603698 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:50.603546    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1982fcfa-c5e2-42b7-8cd4-4f08da150620-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1982fcfa-c5e2-42b7-8cd4-4f08da150620" (UID: "1982fcfa-c5e2-42b7-8cd4-4f08da150620"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:15:50.702209 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:50.702166    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1982fcfa-c5e2-42b7-8cd4-4f08da150620-openshift-service-ca\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:15:50.702209 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:50.702200    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1982fcfa-c5e2-42b7-8cd4-4f08da150620-metrics-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:15:50.702209 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:50.702209    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1982fcfa-c5e2-42b7-8cd4-4f08da150620-maas-api-tls\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:15:50.702209 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:50.702218    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xd567\" (UniqueName: \"kubernetes.io/projected/1982fcfa-c5e2-42b7-8cd4-4f08da150620-kube-api-access-xd567\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Sep 19 11:15:51.242368 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:51.242330    2576 generic.go:356] "Generic (PLEG): container finished" podID="1982fcfa-c5e2-42b7-8cd4-4f08da150620" containerID="1dec620c6ef9bf8904072c064c06b3540db2ced662ff59d7469f2a484d94668b" exitCode=0
Sep 19 11:15:51.242545 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:51.242396    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"
Sep 19 11:15:51.242545 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:51.242422    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz" event={"ID":"1982fcfa-c5e2-42b7-8cd4-4f08da150620","Type":"ContainerDied","Data":"1dec620c6ef9bf8904072c064c06b3540db2ced662ff59d7469f2a484d94668b"}
Sep 19 11:15:51.242545 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:51.242478    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz" event={"ID":"1982fcfa-c5e2-42b7-8cd4-4f08da150620","Type":"ContainerDied","Data":"fdb6c3de975aee712ece39729efc57bfc843ff8806b59ffebaa570d09e713488"}
Sep 19 11:15:51.242545 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:51.242501    2576 scope.go:117] "RemoveContainer" containerID="1dec620c6ef9bf8904072c064c06b3540db2ced662ff59d7469f2a484d94668b"
Sep 19 11:15:51.252587 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:51.252561    2576 scope.go:117] "RemoveContainer" containerID="1dec620c6ef9bf8904072c064c06b3540db2ced662ff59d7469f2a484d94668b"
Sep 19 11:15:51.252976 ip-10-0-142-165 kubenswrapper[2576]: E0919 11:15:51.252955    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dec620c6ef9bf8904072c064c06b3540db2ced662ff59d7469f2a484d94668b\": container with ID starting with 1dec620c6ef9bf8904072c064c06b3540db2ced662ff59d7469f2a484d94668b not found: ID does not exist" containerID="1dec620c6ef9bf8904072c064c06b3540db2ced662ff59d7469f2a484d94668b"
Sep 19 11:15:51.253028 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:51.252987    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dec620c6ef9bf8904072c064c06b3540db2ced662ff59d7469f2a484d94668b"} err="failed to get container status \"1dec620c6ef9bf8904072c064c06b3540db2ced662ff59d7469f2a484d94668b\": rpc error: code = NotFound desc = could not find container \"1dec620c6ef9bf8904072c064c06b3540db2ced662ff59d7469f2a484d94668b\": container with ID starting with 1dec620c6ef9bf8904072c064c06b3540db2ced662ff59d7469f2a484d94668b not found: ID does not exist"
Sep 19 11:15:51.258127 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:51.258091    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"]
Sep 19 11:15:51.261706 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:51.261613    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6599cb9dcd-pr4wz"]
Sep 19 11:15:53.002472 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:15:53.002439    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1982fcfa-c5e2-42b7-8cd4-4f08da150620" path="/var/lib/kubelet/pods/1982fcfa-c5e2-42b7-8cd4-4f08da150620/volumes"
Sep 19 11:16:11.759355 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:11.759324    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-656bb6cb6-lpwbh_6127ea09-2783-4ecd-9c08-8d6814565859/authorino/0.log"
Sep 19 11:16:17.172078 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:17.171403    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7599657744-8c88j_77ce7462-639a-46b0-b250-33ee0966ae6b/manager/0.log"
Sep 19 11:16:17.790057 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:17.790013    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x_42161c4c-6456-45dc-a90f-8623b3a80457/copy-manifests/0.log"
Sep 19 11:16:17.796386 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:17.796359    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x_42161c4c-6456-45dc-a90f-8623b3a80457/manager/0.log"
Sep 19 11:16:18.130257 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:18.130216    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6d474ff8d6-8skgk_7217fcef-611d-47b6-ba5d-e467719042cf/maas-api/0.log"
Sep 19 11:16:18.582337 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:18.582250    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-x4lxv_758ba15e-8cf6-41d9-9f30-579dc0ed6146/postgres/0.log"
Sep 19 11:16:19.319507 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:19.319469    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-656bb6cb6-lpwbh_6127ea09-2783-4ecd-9c08-8d6814565859/authorino/0.log"
Sep 19 11:16:20.577843 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:20.577815    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d97bd96f8-cjbpx_6fb08888-aa5a-4e4d-a9ba-e96f011060f7/kube-auth-proxy/0.log"
Sep 19 11:16:20.907929 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:20.907894    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-fm8xm_fb66578d-ea20-470f-83dc-0c84c6d9ae7e/payload-pre-processing/0.log"
Sep 19 11:16:21.131122 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:21.131085    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6ddf8bf8dd-66fk8_18495139-9a71-4bd7-b279-1d7e6470271f/router/0.log"
Sep 19 11:16:21.662253 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:21.662218    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-g8ldj_33522469-58db-4187-a1d2-aa13594dc9c9/main/0.log"
Sep 19 11:16:27.235302 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:27.235256    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4wp77/must-gather-rrxd9"]
Sep 19 11:16:27.235749 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:27.235734    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1982fcfa-c5e2-42b7-8cd4-4f08da150620" containerName="maas-api"
Sep 19 11:16:27.235796 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:27.235750    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1982fcfa-c5e2-42b7-8cd4-4f08da150620" containerName="maas-api"
Sep 19 11:16:27.235850 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:27.235838    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1982fcfa-c5e2-42b7-8cd4-4f08da150620" containerName="maas-api"
Sep 19 11:16:27.239485 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:27.239457    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4wp77/must-gather-rrxd9"
Sep 19 11:16:27.241934 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:27.241910    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4wp77\"/\"kube-root-ca.crt\""
Sep 19 11:16:27.242891 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:27.242856    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4wp77\"/\"default-dockercfg-5dxml\""
Sep 19 11:16:27.243031 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:27.242857    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4wp77\"/\"openshift-service-ca.crt\""
Sep 19 11:16:27.247684 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:27.247216    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4wp77/must-gather-rrxd9"]
Sep 19 11:16:27.364163 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:27.364121    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gqzn\" (UniqueName: \"kubernetes.io/projected/f1d56f85-c653-479a-8d35-64a521c958aa-kube-api-access-4gqzn\") pod \"must-gather-rrxd9\" (UID: \"f1d56f85-c653-479a-8d35-64a521c958aa\") " pod="openshift-must-gather-4wp77/must-gather-rrxd9"
Sep 19 11:16:27.364369 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:27.364192    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f1d56f85-c653-479a-8d35-64a521c958aa-must-gather-output\") pod \"must-gather-rrxd9\" (UID: \"f1d56f85-c653-479a-8d35-64a521c958aa\") " pod="openshift-must-gather-4wp77/must-gather-rrxd9"
Sep 19 11:16:27.465349 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:27.465302    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f1d56f85-c653-479a-8d35-64a521c958aa-must-gather-output\") pod \"must-gather-rrxd9\" (UID: \"f1d56f85-c653-479a-8d35-64a521c958aa\") " pod="openshift-must-gather-4wp77/must-gather-rrxd9"
Sep 19 11:16:27.465545 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:27.465395    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gqzn\" (UniqueName: \"kubernetes.io/projected/f1d56f85-c653-479a-8d35-64a521c958aa-kube-api-access-4gqzn\") pod \"must-gather-rrxd9\" (UID: \"f1d56f85-c653-479a-8d35-64a521c958aa\") " pod="openshift-must-gather-4wp77/must-gather-rrxd9"
Sep 19 11:16:27.465678 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:27.465658    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f1d56f85-c653-479a-8d35-64a521c958aa-must-gather-output\") pod \"must-gather-rrxd9\" (UID: \"f1d56f85-c653-479a-8d35-64a521c958aa\") " pod="openshift-must-gather-4wp77/must-gather-rrxd9"
Sep 19 11:16:27.474977 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:27.474943    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gqzn\" (UniqueName: \"kubernetes.io/projected/f1d56f85-c653-479a-8d35-64a521c958aa-kube-api-access-4gqzn\") pod \"must-gather-rrxd9\" (UID: \"f1d56f85-c653-479a-8d35-64a521c958aa\") " pod="openshift-must-gather-4wp77/must-gather-rrxd9"
Sep 19 11:16:27.551173 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:27.551071    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4wp77/must-gather-rrxd9"
Sep 19 11:16:27.835224 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:27.835180    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4wp77/must-gather-rrxd9"]
Sep 19 11:16:27.837648 ip-10-0-142-165 kubenswrapper[2576]: W0919 11:16:27.837605    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1d56f85_c653_479a_8d35_64a521c958aa.slice/crio-24f78a6990a5380c1cedd06e8b9d4c5b9bdc25342cd60d71a0a39055cd01fa58 WatchSource:0}: Error finding container 24f78a6990a5380c1cedd06e8b9d4c5b9bdc25342cd60d71a0a39055cd01fa58: Status 404 returned error can't find the container with id 24f78a6990a5380c1cedd06e8b9d4c5b9bdc25342cd60d71a0a39055cd01fa58
Sep 19 11:16:27.839622 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:27.839603    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 11:16:28.442863 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:28.442827    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wp77/must-gather-rrxd9" event={"ID":"f1d56f85-c653-479a-8d35-64a521c958aa","Type":"ContainerStarted","Data":"24f78a6990a5380c1cedd06e8b9d4c5b9bdc25342cd60d71a0a39055cd01fa58"}
Sep 19 11:16:29.453949 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:29.453893    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wp77/must-gather-rrxd9" event={"ID":"f1d56f85-c653-479a-8d35-64a521c958aa","Type":"ContainerStarted","Data":"970bd02b0af0e77388835b97aa6dafce4e934723c0c5cebe0162aeb06f1381bb"}
Sep 19 11:16:29.454503 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:29.453979    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wp77/must-gather-rrxd9" event={"ID":"f1d56f85-c653-479a-8d35-64a521c958aa","Type":"ContainerStarted","Data":"8a2e5be1a2599a5a9772dfe978e5f7a5fff15d4c07ebeba199517b532b4cc222"}
Sep 19 11:16:29.468449 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:29.468374    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4wp77/must-gather-rrxd9" podStartSLOduration=1.643614526 podStartE2EDuration="2.468351444s" totalImagesPullingTime="824.736918ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:16:29.46789461 +0000 UTC m=+2398.964200704" watchObservedRunningTime="2026-09-19 11:16:29.468351444 +0000 UTC m=+2398.964657540"
Sep 19 11:16:30.307577 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:30.307545    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-75sfv_acbcb3f6-b65e-4ee4-b497-e5c913e646d7/global-pull-secret-syncer/0.log"
Sep 19 11:16:30.516950 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:30.516914    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vkrwn_1ec6f61d-3386-407c-8be3-9ba0dfe3045d/konnectivity-agent/0.log"
Sep 19 11:16:30.584668 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:30.584573    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-165.ec2.internal_a889b3717fa0a5037e893154459d9772/haproxy/0.log"
Sep 19 11:16:31.169431 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:31.169401    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 11:16:31.169431 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:31.169417    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 11:16:35.297067 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:35.296655    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-656bb6cb6-lpwbh_6127ea09-2783-4ecd-9c08-8d6814565859/authorino/0.log"
Sep 19 11:16:37.238112 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:37.238076    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f/alertmanager/0.log"
Sep 19 11:16:37.261643 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:37.261545    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f/config-reloader/0.log"
Sep 19 11:16:37.284165 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:37.284127    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f/kube-rbac-proxy-web/0.log"
Sep 19 11:16:37.306237 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:37.306199    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f/kube-rbac-proxy/0.log"
Sep 19 11:16:37.332560 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:37.332413    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f/kube-rbac-proxy-metric/0.log"
Sep 19 11:16:37.357812 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:37.357663    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f/prom-label-proxy/0.log"
Sep 19 11:16:37.384931 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:37.384892    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ebfd45d9-bf0b-40fd-8d7c-3e1c5978722f/init-config-reloader/0.log"
Sep 19 11:16:37.430730 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:37.430688    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-847b8bd8d7-lntx2_0b67d17d-e9f0-4b5b-a15f-760f130cae93/cluster-monitoring-operator/0.log"
Sep 19 11:16:37.562988 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:37.562898    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5984459ff8-pk78w_3bb51bef-73f4-4b3a-9c10-b2eef08ab694/monitoring-plugin/0.log"
Sep 19 11:16:37.767961 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:37.767901    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wz4ff_d38fa806-49b5-4869-87f5-2eeca2e1db69/node-exporter/0.log"
Sep 19 11:16:37.788965 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:37.788935    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wz4ff_d38fa806-49b5-4869-87f5-2eeca2e1db69/kube-rbac-proxy/0.log"
Sep 19 11:16:37.811200 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:37.811170    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wz4ff_d38fa806-49b5-4869-87f5-2eeca2e1db69/init-textfile/0.log"
Sep 19 11:16:37.927881 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:37.927822    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b2a07f45-a695-43dc-aba0-35579c4ca70b/prometheus/0.log"
Sep 19 11:16:37.949241 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:37.949189    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b2a07f45-a695-43dc-aba0-35579c4ca70b/config-reloader/0.log"
Sep 19 11:16:37.972016 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:37.971973    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b2a07f45-a695-43dc-aba0-35579c4ca70b/thanos-sidecar/0.log"
Sep 19 11:16:37.999698 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:37.999663    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b2a07f45-a695-43dc-aba0-35579c4ca70b/kube-rbac-proxy-web/0.log"
Sep 19 11:16:38.026669 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:38.026644    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b2a07f45-a695-43dc-aba0-35579c4ca70b/kube-rbac-proxy/0.log"
Sep 19 11:16:38.052932 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:38.052886    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b2a07f45-a695-43dc-aba0-35579c4ca70b/kube-rbac-proxy-thanos/0.log"
Sep 19 11:16:38.085089 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:38.085034    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b2a07f45-a695-43dc-aba0-35579c4ca70b/init-config-reloader/0.log"
Sep 19 11:16:38.221927 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:38.221828    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f6f8d447c-5xpgf_a5d7b77b-8c41-4139-b39f-a277f9a4bc54/telemeter-client/0.log"
Sep 19 11:16:38.242455 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:38.242408    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f6f8d447c-5xpgf_a5d7b77b-8c41-4139-b39f-a277f9a4bc54/reload/0.log"
Sep 19 11:16:38.267582 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:38.267547    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f6f8d447c-5xpgf_a5d7b77b-8c41-4139-b39f-a277f9a4bc54/kube-rbac-proxy/0.log"
Sep 19 11:16:38.304712 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:38.304681    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-79789845bb-wzxrh_5d46d9af-8b3d-4658-bef1-667d76a7d3f2/thanos-query/0.log"
Sep 19 11:16:38.327297 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:38.327267    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-79789845bb-wzxrh_5d46d9af-8b3d-4658-bef1-667d76a7d3f2/kube-rbac-proxy-web/0.log"
Sep 19 11:16:38.357439 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:38.357405    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-79789845bb-wzxrh_5d46d9af-8b3d-4658-bef1-667d76a7d3f2/kube-rbac-proxy/0.log"
Sep 19 11:16:38.382861 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:38.382824    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-79789845bb-wzxrh_5d46d9af-8b3d-4658-bef1-667d76a7d3f2/prom-label-proxy/0.log"
Sep 19 11:16:38.416376 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:38.416352    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-79789845bb-wzxrh_5d46d9af-8b3d-4658-bef1-667d76a7d3f2/kube-rbac-proxy-rules/0.log"
Sep 19 11:16:38.444831 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:38.444774    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-79789845bb-wzxrh_5d46d9af-8b3d-4658-bef1-667d76a7d3f2/kube-rbac-proxy-metrics/0.log"
Sep 19 11:16:40.829284 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:40.829248    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/1.log"
Sep 19 11:16:40.836309 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:40.836271    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-m25hk_320e460b-f15a-4da5-82eb-f92a0f756180/console-operator/2.log"
Sep 19 11:16:41.376442 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.376402    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-69ccf88565-8cdtz_6ddb962d-a253-4804-a802-a8ec8276ce3f/console/0.log"
Sep 19 11:16:41.438705 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.438661    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"]
Sep 19 11:16:41.444864 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.444827    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"
Sep 19 11:16:41.451002 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.450959    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"]
Sep 19 11:16:41.532072 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.531915    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2-sys\") pod \"perf-node-gather-daemonset-dcxj4\" (UID: \"6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2\") " pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"
Sep 19 11:16:41.532268 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.532091    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2-lib-modules\") pod \"perf-node-gather-daemonset-dcxj4\" (UID: \"6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2\") " pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"
Sep 19 11:16:41.532268 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.532151    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2-proc\") pod \"perf-node-gather-daemonset-dcxj4\" (UID: \"6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2\") " pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"
Sep 19 11:16:41.532268 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.532220    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2-podres\") pod \"perf-node-gather-daemonset-dcxj4\" (UID: \"6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2\") " pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"
Sep 19 11:16:41.532449 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.532276    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qchbh\" (UniqueName: \"kubernetes.io/projected/6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2-kube-api-access-qchbh\") pod \"perf-node-gather-daemonset-dcxj4\" (UID: \"6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2\") " pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"
Sep 19 11:16:41.633621 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.633506    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2-lib-modules\") pod \"perf-node-gather-daemonset-dcxj4\" (UID: \"6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2\") " pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"
Sep 19 11:16:41.633621 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.633582    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2-proc\") pod \"perf-node-gather-daemonset-dcxj4\" (UID: \"6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2\") " pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"
Sep 19 11:16:41.633864 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.633644    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2-podres\") pod \"perf-node-gather-daemonset-dcxj4\" (UID: \"6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2\") " pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"
Sep 19 11:16:41.633864 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.633704    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qchbh\" (UniqueName: \"kubernetes.io/projected/6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2-kube-api-access-qchbh\") pod \"perf-node-gather-daemonset-dcxj4\" (UID: \"6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2\") " pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"
Sep 19 11:16:41.633864 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.633749    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2-sys\") pod \"perf-node-gather-daemonset-dcxj4\" (UID: \"6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2\") " pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"
Sep 19 11:16:41.634028 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.633921    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2-sys\") pod \"perf-node-gather-daemonset-dcxj4\" (UID: \"6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2\") " pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"
Sep 19 11:16:41.634109 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.634070    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2-lib-modules\") pod \"perf-node-gather-daemonset-dcxj4\" (UID: \"6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2\") " pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"
Sep 19 11:16:41.634167 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.634118    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2-proc\") pod \"perf-node-gather-daemonset-dcxj4\" (UID: \"6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2\") " pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"
Sep 19 11:16:41.634215 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.634187    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2-podres\") pod \"perf-node-gather-daemonset-dcxj4\" (UID: \"6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2\") " pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"
Sep 19 11:16:41.642095 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.642037    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qchbh\" (UniqueName: \"kubernetes.io/projected/6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2-kube-api-access-qchbh\") pod \"perf-node-gather-daemonset-dcxj4\" (UID: \"6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2\") " pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"
Sep 19 11:16:41.760722 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:41.760683    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"
Sep 19 11:16:42.069663 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:42.067536    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"]
Sep 19 11:16:42.072982 ip-10-0-142-165 kubenswrapper[2576]: W0919 11:16:42.072926    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6cb0d1a3_dd5c_4dd4_9afd_2d5d0b8752e2.slice/crio-13bbc7d6f29015aa494b261dfb94f149093b27de3c5326452b224fea12c42505 WatchSource:0}: Error finding container 13bbc7d6f29015aa494b261dfb94f149093b27de3c5326452b224fea12c42505: Status 404 returned error can't find the container with id 13bbc7d6f29015aa494b261dfb94f149093b27de3c5326452b224fea12c42505
Sep 19 11:16:42.187516 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:42.187486    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5c4db8bdf9-5nqg9_d526e598-cec6-460b-abcc-e5bee3890d8e/volume-data-source-validator/0.log"
Sep 19 11:16:42.565324 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:42.565279    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4" event={"ID":"6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2","Type":"ContainerStarted","Data":"144abc7117aa1e91f942b3ab593bc1ac82abc6f2838e0a207fe1d7c82746c038"}
Sep 19 11:16:42.565551 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:42.565356    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4" event={"ID":"6cb0d1a3-dd5c-4dd4-9afd-2d5d0b8752e2","Type":"ContainerStarted","Data":"13bbc7d6f29015aa494b261dfb94f149093b27de3c5326452b224fea12c42505"}
Sep 19 11:16:42.566266 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:42.566239    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"
Sep 19 11:16:42.583402 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:42.583324    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4" podStartSLOduration=1.583297709 podStartE2EDuration="1.583297709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 11:16:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:16:42.580101027 +0000 UTC m=+2412.076407119" watchObservedRunningTime="2026-09-19 11:16:42.583297709 +0000 UTC m=+2412.079603802"
Sep 19 11:16:43.419141 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:43.419098    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wf45t_daad582d-23b8-445c-8c99-dec7c11c95a7/dns/0.log"
Sep 19 11:16:43.439747 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:43.439715    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wf45t_daad582d-23b8-445c-8c99-dec7c11c95a7/kube-rbac-proxy/0.log"
Sep 19 11:16:43.485457 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:43.485429    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5f6pp_e96dc328-e8e8-42fb-9db8-6dcbc0f1fe6d/dns-node-resolver/0.log"
Sep 19 11:16:44.120355 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:44.120323    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jvnt2_3bda56d0-e1c8-42c2-8905-9719bd0eace8/node-ca/0.log"
Sep 19 11:16:45.541278 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:45.541251    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d97bd96f8-cjbpx_6fb08888-aa5a-4e4d-a9ba-e96f011060f7/kube-auth-proxy/0.log"
Sep 19 11:16:45.631928 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:45.631890    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-fm8xm_fb66578d-ea20-470f-83dc-0c84c6d9ae7e/payload-pre-processing/0.log"
Sep 19 11:16:45.685576 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:45.685549    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6ddf8bf8dd-66fk8_18495139-9a71-4bd7-b279-1d7e6470271f/router/0.log"
Sep 19 11:16:46.619596 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:46.619563    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g5qhx_bd12ed6b-5433-4b19-9eba-1973e0bd4afa/serve-healthcheck-canary/0.log"
Sep 19 11:16:47.110961 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:47.110906    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-44pw4_2bfbbce9-4493-4ec5-b0d5-d85da7e26e65/insights-operator/0.log"
Sep 19 11:16:47.111457 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:47.111439    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-44pw4_2bfbbce9-4493-4ec5-b0d5-d85da7e26e65/insights-operator/1.log"
Sep 19 11:16:47.197450 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:47.197405    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lch4t_2de2ff6b-99b1-4a41-ba5e-a30034617fac/kube-rbac-proxy/0.log"
Sep 19 11:16:47.218012 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:47.217987    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lch4t_2de2ff6b-99b1-4a41-ba5e-a30034617fac/exporter/0.log"
Sep 19 11:16:47.239102 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:47.239069    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lch4t_2de2ff6b-99b1-4a41-ba5e-a30034617fac/extractor/0.log"
Sep 19 11:16:49.594206 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:49.594174    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4wp77/perf-node-gather-daemonset-dcxj4"
Sep 19 11:16:50.129520 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:50.129480    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7599657744-8c88j_77ce7462-639a-46b0-b250-33ee0966ae6b/manager/0.log"
Sep 19 11:16:50.497684 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:50.497577    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x_42161c4c-6456-45dc-a90f-8623b3a80457/manager/0.log"
Sep 19 11:16:50.519183 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:50.519151    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d7b84ccc8-sgb7x_42161c4c-6456-45dc-a90f-8623b3a80457/copy-manifests/0.log"
Sep 19 11:16:52.065884 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:52.065852    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-749b85b68c-fl7vt_00720b60-9494-4b3a-a68e-7ae372dba963/manager/0.log"
Sep 19 11:16:53.352785 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:53.352754    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d97bd96f8-cjbpx_6fb08888-aa5a-4e4d-a9ba-e96f011060f7/kube-auth-proxy/0.log"
Sep 19 11:16:53.418515 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:53.418476    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-fm8xm_fb66578d-ea20-470f-83dc-0c84c6d9ae7e/payload-pre-processing/0.log"
Sep 19 11:16:53.454461 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:53.454397    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6ddf8bf8dd-66fk8_18495139-9a71-4bd7-b279-1d7e6470271f/router/0.log"
Sep 19 11:16:55.425105 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:55.425071    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d97bd96f8-cjbpx_6fb08888-aa5a-4e4d-a9ba-e96f011060f7/kube-auth-proxy/0.log"
Sep 19 11:16:55.506257 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:55.506221    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-fm8xm_fb66578d-ea20-470f-83dc-0c84c6d9ae7e/payload-pre-processing/0.log"
Sep 19 11:16:55.549992 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:55.549955    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6ddf8bf8dd-66fk8_18495139-9a71-4bd7-b279-1d7e6470271f/router/0.log"
Sep 19 11:16:57.822363 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:57.822332    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-96xsm_26cdce24-be14-4ec9-a4cd-403bde09dd5d/kube-storage-version-migrator-operator/1.log"
Sep 19 11:16:57.823220 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:57.823199    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-96xsm_26cdce24-be14-4ec9-a4cd-403bde09dd5d/kube-storage-version-migrator-operator/0.log"
Sep 19 11:16:59.138946 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:59.138909    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z6nh5_e10a7fca-fab2-40fe-888c-425f875e9a3e/kube-multus-additional-cni-plugins/0.log"
Sep 19 11:16:59.159309 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:59.159284    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z6nh5_e10a7fca-fab2-40fe-888c-425f875e9a3e/egress-router-binary-copy/0.log"
Sep 19 11:16:59.179012 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:59.178980    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z6nh5_e10a7fca-fab2-40fe-888c-425f875e9a3e/cni-plugins/0.log"
Sep 19 11:16:59.199222 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:59.199198    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z6nh5_e10a7fca-fab2-40fe-888c-425f875e9a3e/bond-cni-plugin/0.log"
Sep 19 11:16:59.221262 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:59.221232    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z6nh5_e10a7fca-fab2-40fe-888c-425f875e9a3e/routeoverride-cni/0.log"
Sep 19 11:16:59.240865 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:59.240830    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z6nh5_e10a7fca-fab2-40fe-888c-425f875e9a3e/whereabouts-cni-bincopy/0.log"
Sep 19 11:16:59.260742 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:59.260712    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-z6nh5_e10a7fca-fab2-40fe-888c-425f875e9a3e/whereabouts-cni/0.log"
Sep 19 11:16:59.404154 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:59.404119    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vscrj_a3b17f10-243c-4741-ad8d-feb11c375cbb/kube-multus/0.log"
Sep 19 11:16:59.425868 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:59.425832    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7r6w2_64bed023-f7cc-46f8-b5c0-fbb2e62f56f9/network-metrics-daemon/0.log"
Sep 19 11:16:59.442122 ip-10-0-142-165 kubenswrapper[2576]: I0919 11:16:59.442090    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7r6w2_64bed023-f7cc-46f8-b5c0-fbb2e62f56f9/kube-rbac-proxy/0.log"