Jul 22 01:36:53.165277 ip-10-0-131-80 systemd[1]: Starting Kubernetes Kubelet... Jul 22 01:36:53.613898 ip-10-0-131-80 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 01:36:53.613898 ip-10-0-131-80 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 01:36:53.613898 ip-10-0-131-80 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 01:36:53.613898 ip-10-0-131-80 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 01:36:53.613898 ip-10-0-131-80 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 01:36:53.613898 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.487284 2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 01:36:53.613898 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492097 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 01:36:53.613898 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492114 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 01:36:53.613898 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492119 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 01:36:53.613898 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492122 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 01:36:53.613898 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492125 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 01:36:53.613898 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492128 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 01:36:53.613898 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492131 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 01:36:53.613898 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492133 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 01:36:53.557591 ip-10-0-131-80 systemd[1]: Started Kubernetes Kubelet. Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492136 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492138 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492141 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492144 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492147 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492149 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492154 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492157 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492160 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492163 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492166 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492168 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492171 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492180 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492184 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492186 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492189 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492192 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492195 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 01:36:53.614987 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492197 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492200 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492202 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492205 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492208 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492211 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492213 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492216 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492219 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492222 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492224 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492228 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492232 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492235 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492237 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492240 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492243 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492246 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492248 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492251 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 01:36:53.615679 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492253 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 01:36:53.616408 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492256 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 01:36:53.616408 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492259 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 01:36:53.616408 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492261 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 01:36:53.616408 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492264 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 01:36:53.616408 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492267 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 01:36:53.616408 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492269 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 01:36:53.616408 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492273 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 01:36:53.616408 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492275 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 01:36:53.616408 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492278 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 01:36:53.616408 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492280 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 01:36:53.616408 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492283 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 01:36:53.616408 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492285 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 01:36:53.616408 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492288 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 01:36:53.616408 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492290 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 01:36:53.616408 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492294 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 01:36:53.616408 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492297 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 01:36:53.616408 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492300 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 01:36:53.616408 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492308 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 01:36:53.616408 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492311 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492315 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492317 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492320 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492322 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492325 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492328 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492330 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492333 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492336 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492339 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492342 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492344 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492347 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492349 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492352 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492355 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492358 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492360 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492363 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 01:36:53.617191 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492366 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492368 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492371 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492374 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492377 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492379 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492381 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492384 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492387 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492389 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492391 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492394 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.492396 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494343 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494362 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494367 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494372 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494377 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494382 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494386 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 01:36:53.617892 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494439 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494467 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494473 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494478 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494492 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494506 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494509 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494512 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494515 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494649 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494652 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494655 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494658 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494661 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494664 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494667 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494670 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494673 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494676 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494679 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 01:36:53.618628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494682 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 01:36:53.619308 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494684 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 01:36:53.619308 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494687 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 01:36:53.619308 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494690 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 01:36:53.619308 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494692 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 01:36:53.619308 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494695 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 01:36:53.619308 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494697 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 01:36:53.619308 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494700 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 01:36:53.619308 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494702 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 01:36:53.619308 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494705 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 01:36:53.619308 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494708 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 01:36:53.619308 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494712 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 01:36:53.619308 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494715 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 01:36:53.619308 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494718 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 01:36:53.619308 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494720 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 01:36:53.619308 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494731 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 01:36:53.619308 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494735 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 01:36:53.619308 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494738 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 01:36:53.619308 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494740 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 01:36:53.619308 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494744 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494747 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494750 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494752 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494755 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494758 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494761 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494763 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494769 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494771 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494774 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494777 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494780 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494782 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494786 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494789 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494791 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494794 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494797 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494802 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 01:36:53.620044 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494806 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 01:36:53.620971 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494811 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 01:36:53.620971 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494814 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 01:36:53.620971 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494817 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 01:36:53.620971 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494820 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 01:36:53.620971 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494823 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 01:36:53.620971 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494826 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 01:36:53.620971 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494829 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 01:36:53.620971 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494832 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 01:36:53.620971 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494834 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 01:36:53.620971 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494837 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 01:36:53.620971 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494842 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 01:36:53.620971 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494845 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 01:36:53.620971 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494848 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 01:36:53.620971 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494851 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 01:36:53.620971 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494853 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 22 01:36:53.620971 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494856 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 01:36:53.620971 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494858 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 01:36:53.620971 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494861 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 01:36:53.620971 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494863 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 01:36:53.621617 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494866 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 01:36:53.621617 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494869 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 01:36:53.621617 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494871 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 01:36:53.621617 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494874 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 01:36:53.621617 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494876 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 01:36:53.621617 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494879 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 01:36:53.621617 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494881 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 01:36:53.621617 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494885 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 01:36:53.621617 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494889 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 01:36:53.621617 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494892 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 01:36:53.621617 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494895 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 01:36:53.621617 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494898 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 01:36:53.621617 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.494901 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 01:36:53.621617 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495326 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 01:36:53.621617 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495331 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 01:36:53.621617 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495334 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 01:36:53.621617 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495337 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 01:36:53.621617 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495341 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 01:36:53.621617 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495345 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 01:36:53.622206 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495349 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 01:36:53.622206 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495352 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 01:36:53.622206 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495355 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 01:36:53.622206 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495358 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 01:36:53.622206 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495361 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 01:36:53.622206 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495364 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 01:36:53.622206 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495367 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 01:36:53.622206 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495369 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 01:36:53.622206 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495373 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 01:36:53.622206 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495376 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 01:36:53.622206 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495379 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 01:36:53.622206 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495382 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 01:36:53.622206 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495384 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 01:36:53.622206 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495387 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 01:36:53.622206 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495390 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 01:36:53.622206 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495393 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 01:36:53.622206 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495396 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 01:36:53.622206 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495398 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 01:36:53.622206 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495401 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495404 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495407 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495410 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495412 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495415 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495417 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495420 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495423 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495426 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495429 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495431 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495433 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495437 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495439 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495442 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495445 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495448 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495450 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495453 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 01:36:53.622799 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495457 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495460 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495462 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495465 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495468 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495471 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495473 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495476 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495478 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495481 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495483 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495486 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495488 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495491 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495493 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495496 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495498 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495501 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495503 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495506 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 01:36:53.623339 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495509 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 01:36:53.624001 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495511 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 01:36:53.624001 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495514 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 01:36:53.624001 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495517 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 01:36:53.624001 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495519 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 01:36:53.624001 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495524 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 01:36:53.624001 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495527 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 01:36:53.624001 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495530 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 01:36:53.624001 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495533 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 01:36:53.624001 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495535 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 01:36:53.624001 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495538 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 01:36:53.624001 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495541 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 01:36:53.624001 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495543 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 01:36:53.624001 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495546 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 01:36:53.624001 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495548 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 01:36:53.624001 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495551 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 01:36:53.624001 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495553 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 01:36:53.624001 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495559 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 01:36:53.624001 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495562 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 01:36:53.624001 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495564 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495567 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495569 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495572 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495574 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495577 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495579 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495581 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495584 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495586 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495589 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495591 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495594 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495596 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495603 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.495606 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495696 2559 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495704 2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495711 2559 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495716 2559 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495720 2559 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 01:36:53.624557 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495723 2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495727 2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495732 2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495736 2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495739 2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495743 2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495746 2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495749 2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495752 2559 flags.go:64] FLAG: --cgroup-root="" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495757 2559 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495760 2559 flags.go:64] FLAG: --client-ca-file="" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495763 2559 flags.go:64] FLAG: --cloud-provider="external" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495766 2559 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495771 2559 flags.go:64] FLAG: --cluster-domain="" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495774 2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495777 2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495780 2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495783 2559 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495787 2559 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495790 2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495793 2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495797 2559 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495799 2559 flags.go:64] FLAG: --contention-profiling="false" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495802 2559 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 01:36:53.625334 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495805 2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495810 2559 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495812 2559 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495817 2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495820 2559 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495823 2559 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495826 2559 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495829 2559 flags.go:64] FLAG: --enable-server="true" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495831 2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495836 2559 flags.go:64] FLAG: --event-burst="100" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495838 2559 flags.go:64] FLAG: --event-qps="50" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495841 2559 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495844 2559 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495847 2559 flags.go:64] FLAG: --eviction-hard="" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495851 2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495853 2559 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495856 2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495861 2559 flags.go:64] FLAG: --eviction-soft="" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495864 2559 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495867 2559 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495870 2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495873 2559 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495876 2559 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495879 2559 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495881 2559 flags.go:64] FLAG: --feature-gates="" Jul 22 01:36:53.626173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495885 2559 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495888 2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495891 2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495894 2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495897 2559 flags.go:64] FLAG: --healthz-port="10248" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495900 2559 flags.go:64] FLAG: --help="false" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495902 2559 flags.go:64] FLAG: --hostname-override="ip-10-0-131-80.ec2.internal" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495919 2559 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495924 2559 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495927 2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495930 2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495933 2559 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495936 2559 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495939 2559 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495942 2559 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495945 2559 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495948 2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495952 2559 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495955 2559 flags.go:64] FLAG: --kube-reserved="" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495957 2559 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495960 2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495963 2559 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495966 2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495969 2559 flags.go:64] FLAG: --lock-file="" Jul 22 01:36:53.627193 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495973 2559 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495976 2559 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495979 2559 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495984 2559 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495988 2559 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495990 2559 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495993 2559 flags.go:64] FLAG: --logging-format="text" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.495996 2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496000 2559 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496002 2559 flags.go:64] FLAG: --manifest-url="" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496005 2559 flags.go:64] FLAG: --manifest-url-header="" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496009 2559 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496012 2559 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496016 2559 flags.go:64] FLAG: --max-pods="110" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496019 2559 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496022 2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496026 2559 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496029 2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496032 2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496035 2559 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496038 2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496044 2559 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496047 2559 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496050 2559 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 01:36:53.629101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496053 2559 flags.go:64] FLAG: --pod-cidr="" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496056 2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496062 2559 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496064 2559 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496067 2559 flags.go:64] FLAG: --pods-per-core="0" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496070 2559 flags.go:64] FLAG: --port="10250" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496073 2559 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496076 2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04c26169cb309d8f1" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496082 2559 flags.go:64] FLAG: --qos-reserved="" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496085 2559 flags.go:64] FLAG: --read-only-port="10255" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496088 2559 flags.go:64] FLAG: --register-node="true" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496091 2559 flags.go:64] FLAG: --register-with-taints="" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496095 2559 flags.go:64] FLAG: --registry-burst="10" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496098 2559 flags.go:64] FLAG: --registry-qps="5" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496101 2559 flags.go:64] FLAG: --reserved-cpus="" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496104 2559 flags.go:64] FLAG: --reserved-memory="" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496108 2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496111 2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496114 2559 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496117 2559 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496120 2559 flags.go:64] FLAG: --runonce="false" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496123 2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496126 2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496129 2559 flags.go:64] FLAG: --seccomp-default="false" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496132 2559 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 01:36:55.731771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496136 2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496139 2559 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496142 2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496145 2559 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496148 2559 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496150 2559 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496153 2559 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496156 2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496159 2559 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496162 2559 flags.go:64] FLAG: --system-cgroups="" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496164 2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496170 2559 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496172 2559 flags.go:64] FLAG: --tls-cert-file="" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496175 2559 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496179 2559 flags.go:64] FLAG: --tls-min-version="" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496184 2559 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496186 2559 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496189 2559 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496192 2559 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496195 2559 flags.go:64] FLAG: --v="2" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496199 2559 flags.go:64] FLAG: --version="false" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496203 2559 flags.go:64] FLAG: --vmodule="" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496207 2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496211 2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496325 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 01:36:55.755455 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496329 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496332 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496335 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496337 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496340 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496342 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496345 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496349 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496351 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496355 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496357 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496360 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496363 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496365 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496368 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496370 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496372 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496375 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496377 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496380 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 01:36:55.756382 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496382 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496385 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496389 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496392 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496394 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496396 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496399 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496401 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496404 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496407 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496410 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496412 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496415 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496417 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496420 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496422 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496425 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496427 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496430 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496433 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 01:36:55.757186 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496436 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 01:36:55.757787 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496438 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 01:36:55.757787 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496441 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 01:36:55.757787 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496444 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 01:36:55.757787 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496446 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 01:36:55.757787 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496449 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 01:36:55.757787 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496451 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 01:36:55.757787 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496454 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 01:36:55.757787 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496457 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 01:36:55.757787 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496461 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 01:36:55.757787 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496464 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 01:36:55.757787 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496468 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 01:36:55.757787 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496471 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 01:36:55.757787 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496474 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 01:36:55.757787 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496478 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 01:36:55.757787 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496481 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 01:36:55.757787 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496483 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 01:36:55.757787 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496486 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 01:36:55.757787 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496488 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 22 01:36:55.757787 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496490 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 01:36:55.758573 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496493 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 01:36:55.758573 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496495 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 01:36:55.758573 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496498 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 01:36:55.758573 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496501 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 01:36:55.758573 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496503 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 01:36:55.758573 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496506 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 01:36:55.758573 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496508 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 01:36:55.758573 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496511 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 01:36:55.758573 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496513 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 01:36:55.758573 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496516 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 01:36:55.758573 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496519 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 01:36:55.758573 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496522 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 01:36:55.758573 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496525 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 01:36:55.758573 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496527 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 01:36:55.758573 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496530 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 01:36:55.758573 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496532 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 01:36:55.758573 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496535 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 01:36:55.758573 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496537 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 01:36:55.758573 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496540 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 01:36:55.759171 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496542 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 01:36:55.759171 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496545 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 01:36:55.759171 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496547 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 01:36:55.759171 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496550 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 01:36:55.759171 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496552 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 01:36:55.759171 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496555 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 01:36:55.759171 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496557 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 01:36:55.759171 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496564 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 01:36:55.759171 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496566 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 01:36:55.759171 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496568 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 01:36:55.759171 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496571 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 01:36:55.759171 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496575 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 01:36:55.759171 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496578 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 01:36:55.759171 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496581 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 01:36:55.759171 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496584 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 01:36:55.759171 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496586 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 01:36:55.759171 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496589 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 01:36:55.759171 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496592 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 01:36:55.759171 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.496594 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 01:36:55.759759 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.496600 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 01:36:55.759759 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.503565 2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 01:36:55.759759 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.503584 2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 01:36:55.759759 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.504173 2559 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 01:36:55.759759 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.504181 2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 01:36:55.759759 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504260 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 01:36:55.759759 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504266 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 01:36:55.759759 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504270 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 01:36:55.759759 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504273 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 01:36:55.759759 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504275 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 01:36:55.759759 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504278 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 01:36:55.759759 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504281 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 01:36:55.759759 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504283 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 01:36:55.759759 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504286 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 22 01:36:55.759759 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504289 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 01:36:55.760254 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504292 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 01:36:55.760254 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504295 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 01:36:55.760254 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504297 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 01:36:55.760254 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504300 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 01:36:55.760254 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504303 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 01:36:55.760254 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504305 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 01:36:55.760254 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504311 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 01:36:55.760254 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504314 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 01:36:55.760254 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504317 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 01:36:55.760254 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504319 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 01:36:55.760254 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504323 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 01:36:55.760254 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504328 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 01:36:55.760254 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504331 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 01:36:55.760254 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504334 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 01:36:55.760254 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504337 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 01:36:55.760254 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504340 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 01:36:55.760254 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504343 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 01:36:55.760254 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504346 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 01:36:55.760254 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504349 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504353 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504356 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504359 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504361 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504364 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504367 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504372 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504375 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504377 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504380 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504382 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504385 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504387 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504390 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504392 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504395 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504399 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504402 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504405 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 01:36:55.760797 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504409 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504411 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504414 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504416 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504419 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504421 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504424 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504427 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504429 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504433 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504435 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504438 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504440 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504443 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504446 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504449 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504463 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504467 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504470 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504475 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 01:36:55.761378 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504477 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 01:36:55.761950 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504480 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 01:36:55.761950 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504484 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 01:36:55.761950 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504487 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 01:36:55.761950 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504490 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 01:36:55.761950 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504493 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 01:36:55.761950 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504496 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 01:36:55.761950 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504499 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 01:36:55.761950 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504501 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 01:36:55.761950 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504504 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 01:36:55.761950 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504506 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 01:36:55.761950 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504509 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 01:36:55.761950 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504513 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 01:36:55.761950 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504516 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 01:36:55.761950 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504518 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 01:36:55.761950 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504521 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 01:36:55.761950 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504523 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 01:36:55.761950 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504526 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 01:36:55.761950 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504528 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 01:36:55.761950 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504530 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 01:36:55.762518 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504533 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 01:36:55.762518 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504536 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 01:36:55.762518 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504538 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 01:36:55.762518 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504541 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 01:36:55.762518 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504543 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 01:36:55.762518 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504546 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 01:36:55.762518 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504548 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 01:36:55.762518 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504553 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 01:36:55.762518 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504555 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 01:36:55.762518 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504558 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 01:36:55.762518 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504560 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 01:36:55.762518 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.504567 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 01:36:55.762518 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504692 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 01:36:55.762518 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504697 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 01:36:55.762518 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504700 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 01:36:55.762940 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504703 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 01:36:55.762940 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504705 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 01:36:55.762940 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504708 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 01:36:55.762940 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504710 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 01:36:55.762940 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504713 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 01:36:55.762940 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504716 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 01:36:55.762940 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504718 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 01:36:55.762940 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504721 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 01:36:55.762940 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504723 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 01:36:55.762940 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504727 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 01:36:55.762940 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504730 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 01:36:55.762940 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504732 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 01:36:55.762940 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504735 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 01:36:55.762940 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504737 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 01:36:55.762940 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504740 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 01:36:55.762940 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504742 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 01:36:55.762940 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504745 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 01:36:55.762940 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504747 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 01:36:55.762940 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504750 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504753 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504755 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504758 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504760 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504763 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504766 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504769 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504772 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504774 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504777 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504780 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504783 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504792 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504796 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504799 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504802 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504804 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504806 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504809 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 01:36:55.763545 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504812 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 01:36:55.764239 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504814 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 01:36:55.764239 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504817 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 01:36:55.764239 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504821 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 01:36:55.764239 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504842 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 01:36:55.764239 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504844 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 01:36:55.764239 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504847 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 01:36:55.764239 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504850 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 01:36:55.764239 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504852 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 01:36:55.764239 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504855 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 01:36:55.764239 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504857 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 01:36:55.764239 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504861 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 01:36:55.764239 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504863 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 01:36:55.764239 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504867 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 01:36:55.764239 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504871 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 01:36:55.764239 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504874 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 01:36:55.764239 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504877 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 01:36:55.764239 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504880 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 01:36:55.764239 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504882 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 01:36:55.764239 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504885 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504889 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504892 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504895 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504898 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504901 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504921 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504925 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504928 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504931 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504933 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504936 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504939 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504942 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504944 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504947 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504951 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504954 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504956 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504959 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 01:36:55.764835 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504961 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 01:36:55.765383 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504965 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 01:36:55.765383 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504969 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 22 01:36:55.765383 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504972 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 01:36:55.765383 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504974 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 01:36:55.765383 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504977 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 01:36:55.765383 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504980 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 01:36:55.765383 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504982 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 01:36:55.765383 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504985 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 01:36:55.765383 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504988 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 01:36:55.765383 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504990 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 01:36:55.765383 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504994 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 01:36:55.765383 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.504998 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 01:36:55.765383 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.505001 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 01:36:55.765383 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.505005 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 01:36:55.765383 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.505008 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 01:36:55.765383 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.505012 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 01:36:55.765383 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.505015 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 01:36:55.765383 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:53.505017 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 01:36:55.765900 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.505022 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 01:36:55.765900 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.505172 2559 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 01:36:55.765900 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.507125 2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 01:36:55.765900 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.508302 2559 server.go:1019] "Starting client certificate rotation" Jul 22 01:36:55.765900 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.508413 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 01:36:55.765900 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.508463 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 01:36:55.765900 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.510339 2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 01:36:55.765900 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.513413 2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 01:36:55.765900 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.524686 2559 log.go:25] "Validated CRI v1 runtime API" Jul 22 01:36:55.765900 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.526882 2559 log.go:25] "Validated CRI v1 image API" Jul 22 01:36:55.765900 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.527713 2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 01:36:55.765900 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.533755 2559 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 85b495df-9272-4ea4-95d8-92eb78b4426a:/dev/nvme0n1p3 c6b26fa2-d07f-41c3-a407-556e176cdb97:/dev/nvme0n1p4] Jul 22 01:36:55.766254 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.533773 2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 01:36:55.766254 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.538807 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 01:36:55.975526 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.539601 2559 manager.go:216] Machine: {Timestamp:2026-07-22 01:36:53.538454211 +0000 UTC m=+0.274140990 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3175928 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec240cf8354bdec0f4420f9a4e332e3c SystemUUID:ec240cf8-354b-dec0-f442-0f9a4e332e3c BootID:40e90873-1b20-4e1b-ae49-3abc63f4608f Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9e:e6:6b:b7:e5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9e:e6:6b:b7:e5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:76:8d:48:d3:0a:32 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 01:36:55.975526 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.539703 2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 01:36:55.975526 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.539835 2559 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 01:36:55.975526 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.542117 2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 01:36:55.975526 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.542151 2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-80.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 01:36:55.975526 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.542719 2559 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 01:36:55.975526 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.542731 2559 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 01:36:55.975526 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.542742 2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 01:36:55.975526 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.543205 2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 01:36:55.975526 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.544249 2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 01:36:55.975526 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.547602 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 22 01:36:55.975526 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.547730 2559 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 01:36:55.975526 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.547753 2559 kubelet.go:486] "Attempting to sync node with API server" Jul 22 01:36:55.975526 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.547764 2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 01:36:55.975526 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.547776 2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 01:36:55.975526 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.547785 2559 kubelet.go:398] "Adding apiserver pod source" Jul 22 01:36:55.975526 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.547796 2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 01:36:55.975526 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.548163 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.548179 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.549919 2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.551683 2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.551712 2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.552582 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.552596 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.552602 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.552607 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.552617 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.552622 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.552631 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.552636 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.552643 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.552649 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.552660 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.552665 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.553240 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.553248 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.556608 2559 server.go:1268] "Started kubelet" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.556695 2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.556726 2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.556773 2559 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.558194 2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.560238 2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-80.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:53.560241 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:53.560236 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-80.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.560880 2559 server.go:310] "Adding debug handlers to kubelet server" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:53.563582 2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:53.563628 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-80.ec2.internal.18c478d6294986ee default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-80.ec2.internal,UID:ip-10-0-131-80.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-80.ec2.internal,},FirstTimestamp:2026-07-22 01:36:53.55657803 +0000 UTC m=+0.292264808,LastTimestamp:2026-07-22 01:36:53.55657803 +0000 UTC m=+0.292264808,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-80.ec2.internal,}" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.565743 2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.565793 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:53.565825 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-80.ec2.internal\" not found" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.566285 2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.566385 2559 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.566403 2559 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.566545 2559 reconstruct.go:97] "Volume reconstruction finished" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.566555 2559 reconciler.go:29] "Reconciler: start to sync state" Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.566866 2559 factory.go:153] Registering CRI-O factory Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.566882 2559 factory.go:223] Registration of the crio container factory successfully Jul 22 01:36:55.976178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.566963 2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.566973 2559 factory.go:55] Registering systemd factory Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.566981 2559 factory.go:223] Registration of the systemd container factory successfully Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.567003 2559 factory.go:103] Registering Raw factory Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.567016 2559 manager.go:1196] Started watching for new ooms in manager Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.567342 2559 manager.go:349] Starting recovery of all containers Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:53.572976 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:53.573034 2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-80.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.579954 2559 manager.go:354] Recovery completed Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.585692 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.587435 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasSufficientMemory" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.587472 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.587482 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasSufficientPID" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.588053 2559 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.588062 2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.588064 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.588078 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.588449 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dckxs" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.588716 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasSufficientMemory" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.588739 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.588750 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasSufficientPID" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:53.589742 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-80.ec2.internal.18c478d62b20b6dc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-80.ec2.internal,UID:ip-10-0-131-80.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-80.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-80.ec2.internal,},FirstTimestamp:2026-07-22 01:36:53.587457756 +0000 UTC m=+0.323144534,LastTimestamp:2026-07-22 01:36:53.587457756 +0000 UTC m=+0.323144534,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-80.ec2.internal,}" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.590086 2559 policy_none.go:49] "None policy: Start" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.590100 2559 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.590110 2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.591955 2559 policy_none.go:47] "Start" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.596389 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dckxs" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:53.601276 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-80.ec2.internal.18c478d62b20ff83 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-80.ec2.internal,UID:ip-10-0-131-80.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-80.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-80.ec2.internal,},FirstTimestamp:2026-07-22 01:36:53.587476355 +0000 UTC m=+0.323163132,LastTimestamp:2026-07-22 01:36:53.587476355 +0000 UTC m=+0.323163132,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-80.ec2.internal,}" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.631626 2559 manager.go:337] "Starting Device Plugin manager" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:53.631686 2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.631699 2559 server.go:87] "Starting device plugin registration server" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.632009 2559 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.632022 2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.632106 2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 01:36:56.183124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.632214 2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.632221 2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:53.632737 2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:53.632784 2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-80.ec2.internal\" not found" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.711446 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.712654 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.712693 2559 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.712723 2559 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:53.712764 2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.716009 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.732540 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.733574 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasSufficientMemory" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.733604 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.733613 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasSufficientPID" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.733638 2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-80.ec2.internal" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.742610 2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-80.ec2.internal" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:53.742636 2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-80.ec2.internal\": node \"ip-10-0-131-80.ec2.internal\" not found" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:53.768254 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-80.ec2.internal\" not found" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.813043 2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-80.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal"] Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.813157 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.814151 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasSufficientMemory" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.814180 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:53.814190 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasSufficientPID" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.722836 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.722987 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-80.ec2.internal" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.723015 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.723652 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasSufficientMemory" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.723673 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.723684 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasSufficientPID" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.723656 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasSufficientMemory" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.723746 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.723759 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasSufficientPID" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.725923 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.725955 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.726703 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasSufficientMemory" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.726729 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.726742 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasSufficientPID" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:55.765407 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-80.ec2.internal\" not found" node="ip-10-0-131-80.ec2.internal" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:55.770034 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-80.ec2.internal\" not found" node="ip-10-0-131-80.ec2.internal" Jul 22 01:36:56.184905 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:55.775218 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-80.ec2.internal\" not found" Jul 22 01:36:56.185955 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.775280 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ae3d302a83e4b95708739b07d02ee6e8-config\") pod \"kube-apiserver-proxy-ip-10-0-131-80.ec2.internal\" (UID: \"ae3d302a83e4b95708739b07d02ee6e8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-80.ec2.internal" Jul 22 01:36:56.185955 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.775324 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f63d47aab8d4135421bfb4b85518502-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal\" (UID: \"9f63d47aab8d4135421bfb4b85518502\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal" Jul 22 01:36:56.185955 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.775355 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f63d47aab8d4135421bfb4b85518502-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal\" (UID: \"9f63d47aab8d4135421bfb4b85518502\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal" Jul 22 01:36:56.185955 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:55.876027 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-80.ec2.internal\" not found" Jul 22 01:36:56.185955 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.876049 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ae3d302a83e4b95708739b07d02ee6e8-config\") pod \"kube-apiserver-proxy-ip-10-0-131-80.ec2.internal\" (UID: \"ae3d302a83e4b95708739b07d02ee6e8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-80.ec2.internal" Jul 22 01:36:56.185955 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.876077 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f63d47aab8d4135421bfb4b85518502-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal\" (UID: \"9f63d47aab8d4135421bfb4b85518502\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal" Jul 22 01:36:56.185955 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.876095 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f63d47aab8d4135421bfb4b85518502-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal\" (UID: \"9f63d47aab8d4135421bfb4b85518502\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal" Jul 22 01:36:56.185955 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.876139 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f63d47aab8d4135421bfb4b85518502-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal\" (UID: \"9f63d47aab8d4135421bfb4b85518502\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal" Jul 22 01:36:56.185955 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.876148 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f63d47aab8d4135421bfb4b85518502-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal\" (UID: \"9f63d47aab8d4135421bfb4b85518502\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal" Jul 22 01:36:56.185955 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:55.876148 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ae3d302a83e4b95708739b07d02ee6e8-config\") pod \"kube-apiserver-proxy-ip-10-0-131-80.ec2.internal\" (UID: \"ae3d302a83e4b95708739b07d02ee6e8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-80.ec2.internal" Jul 22 01:36:56.185955 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:55.976951 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-80.ec2.internal\" not found" Jul 22 01:36:56.185955 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:56.068208 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-80.ec2.internal" Jul 22 01:36:56.185955 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:56.073093 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal" Jul 22 01:36:56.185955 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:56.077712 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-80.ec2.internal\" not found" Jul 22 01:36:56.185955 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:56.178145 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-80.ec2.internal\" not found" Jul 22 01:36:56.332273 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:56.278538 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-80.ec2.internal\" not found" Jul 22 01:36:56.332273 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:56.307751 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 01:36:56.439638 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:56.379352 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-80.ec2.internal\" not found" Jul 22 01:36:56.439638 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:56.414533 2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 01:36:56.439638 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:56.414702 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 01:36:56.439638 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:56.414724 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 01:36:56.551949 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:56.473993 2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 01:36:56.551949 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:56.479691 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-80.ec2.internal\" not found" Jul 22 01:36:56.551949 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:56.507334 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 01:31:55 +0000 UTC" deadline="2028-04-10 08:57:55.897401714 +0000 UTC" Jul 22 01:36:56.551949 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:56.507383 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15079h20m59.390021999s" Jul 22 01:36:56.555174 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:56.554613 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 01:36:56.558917 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:56.556430 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasSufficientMemory" Jul 22 01:36:56.558917 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:56.556479 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 01:36:56.558917 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:56.556492 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeHasSufficientPID" Jul 22 01:36:56.668752 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:56.579919 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-80.ec2.internal\" not found" Jul 22 01:36:56.681150 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:56.680607 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-80.ec2.internal\" not found" Jul 22 01:36:56.786599 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:56.781531 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-80.ec2.internal\" not found" Jul 22 01:36:56.913151 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:56.881954 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-80.ec2.internal\" not found" Jul 22 01:36:57.036591 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:56.982589 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-80.ec2.internal\" not found" Jul 22 01:36:57.036591 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.017894 2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 01:36:57.155738 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.072932 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-80.ec2.internal" Jul 22 01:36:57.155738 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.084564 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 01:36:57.155738 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.085534 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal" Jul 22 01:36:57.155738 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.100172 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.456590 2559 apiserver.go:52] "Watching apiserver" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.467311 2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.467677 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-tlbhv","kube-system/konnectivity-agent-xtb6t","openshift-cluster-node-tuning-operator/tuned-blbsx","openshift-multus/multus-additional-cni-plugins-9dqvw","openshift-multus/multus-phv9v","openshift-multus/network-metrics-daemon-878hc","openshift-network-diagnostics/network-check-target-zrkr9","openshift-ovn-kubernetes/ovnkube-node-7gfnj","kube-system/kube-apiserver-proxy-ip-10-0-131-80.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr","openshift-dns/node-resolver-zvt6m","openshift-image-registry/node-ca-rs2vg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal","openshift-network-operator/iptables-alerter-hz2w7"] Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.470329 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:57.470416 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrkr9" podUID="4326c7c1-88b7-4ead-be08-a2a2e2497a64" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.472727 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.472862 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zvt6m" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.474943 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hz2w7" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.475499 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f8bfk\"" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.475753 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.475840 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.477033 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.477073 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.477131 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.477209 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.477517 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-29tgn\"" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.479186 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.479763 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.479770 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.479775 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.479785 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.479816 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-f9mlk\"" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.479855 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.481192 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.481290 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-s8r7s\"" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.481405 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:57.481472 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-878hc" podUID="8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.483563 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rs2vg" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.485125 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzv4z\" (UniqueName: \"kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z\") pod \"network-check-target-zrkr9\" (UID: \"4326c7c1-88b7-4ead-be08-a2a2e2497a64\") " pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:36:57.494891 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.485146 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/acaa1746-a8a5-4517-9299-9a502758b7a4-hosts-file\") pod \"node-resolver-zvt6m\" (UID: \"acaa1746-a8a5-4517-9299-9a502758b7a4\") " pod="openshift-dns/node-resolver-zvt6m" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.485168 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acaa1746-a8a5-4517-9299-9a502758b7a4-tmp-dir\") pod \"node-resolver-zvt6m\" (UID: \"acaa1746-a8a5-4517-9299-9a502758b7a4\") " pod="openshift-dns/node-resolver-zvt6m" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.485188 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/886c82f1-624b-42ba-83fe-339fb2898a20-iptables-alerter-script\") pod \"iptables-alerter-hz2w7\" (UID: \"886c82f1-624b-42ba-83fe-339fb2898a20\") " pod="openshift-network-operator/iptables-alerter-hz2w7" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.485225 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/886c82f1-624b-42ba-83fe-339fb2898a20-host-slash\") pod \"iptables-alerter-hz2w7\" (UID: \"886c82f1-624b-42ba-83fe-339fb2898a20\") " pod="openshift-network-operator/iptables-alerter-hz2w7" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.485249 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsqkb\" (UniqueName: \"kubernetes.io/projected/886c82f1-624b-42ba-83fe-339fb2898a20-kube-api-access-tsqkb\") pod \"iptables-alerter-hz2w7\" (UID: \"886c82f1-624b-42ba-83fe-339fb2898a20\") " pod="openshift-network-operator/iptables-alerter-hz2w7" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.485266 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nnqz\" (UniqueName: \"kubernetes.io/projected/acaa1746-a8a5-4517-9299-9a502758b7a4-kube-api-access-8nnqz\") pod \"node-resolver-zvt6m\" (UID: \"acaa1746-a8a5-4517-9299-9a502758b7a4\") " pod="openshift-dns/node-resolver-zvt6m" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.485573 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.485698 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.485705 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.485723 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.486082 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mgwrh\"" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.486110 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.487830 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.487923 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xtb6t" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.488037 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.488146 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.488627 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qmr5c\"" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.490115 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:57.490171 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbhv" podUID="51b9bc4d-f596-4c94-9337-1428391e8aef" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.490730 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vhfhg\"" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.490793 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.491171 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 01:36:57.496435 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.492406 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.499330 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.497304 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 01:36:57.499330 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.497628 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 01:36:57.499330 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.497646 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 01:36:57.499330 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.497662 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.499330 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.497614 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-f2jvm\"" Jul 22 01:36:57.499330 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.497601 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 01:36:57.499330 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.497627 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 01:36:57.499330 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.497244 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 01:36:57.500556 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.500114 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 01:36:57.500556 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.500127 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6hgjf\"" Jul 22 01:36:57.502914 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.500688 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 01:36:57.522126 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.522098 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zm5fs" Jul 22 01:36:57.528818 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.528800 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zm5fs" Jul 22 01:36:57.573470 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.573446 2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 01:36:57.585755 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.585731 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-node-log\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.585755 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.585757 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-device-dir\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.585927 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.585772 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-var-lib-kubelet\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.585927 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.585788 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/efeab439-38f6-4ebe-8a23-0c8f6d34e743-system-cni-dir\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.585927 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.585806 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-kubernetes\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.585927 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.585844 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-systemd\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.585927 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.585889 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-cnibin\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.585927 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.585918 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-multus-socket-dir-parent\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.586112 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.585942 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nnqz\" (UniqueName: \"kubernetes.io/projected/acaa1746-a8a5-4517-9299-9a502758b7a4-kube-api-access-8nnqz\") pod \"node-resolver-zvt6m\" (UID: \"acaa1746-a8a5-4517-9299-9a502758b7a4\") " pod="openshift-dns/node-resolver-zvt6m" Jul 22 01:36:57.586112 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.585958 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-run-ovn-kubernetes\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.586112 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.585977 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/efeab439-38f6-4ebe-8a23-0c8f6d34e743-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.586112 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586002 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-sysconfig\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.586112 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586018 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-multus-conf-dir\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.586112 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586033 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-run-netns\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.586112 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586049 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-run-openvswitch\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.586112 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586062 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-log-socket\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.586112 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586081 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-cni-netd\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.586112 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586102 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72d8f397-d16d-446b-9d87-c2064ba99f21-ovnkube-config\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.586414 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586128 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-sysctl-conf\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.586414 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586145 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-multus-cni-dir\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.586414 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586174 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/26965363-ec8e-46d5-8647-25aa21dd2896-cni-binary-copy\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.586414 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586193 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/886c82f1-624b-42ba-83fe-339fb2898a20-iptables-alerter-script\") pod \"iptables-alerter-hz2w7\" (UID: \"886c82f1-624b-42ba-83fe-339fb2898a20\") " pod="openshift-network-operator/iptables-alerter-hz2w7" Jul 22 01:36:57.586414 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586244 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/51b9bc4d-f596-4c94-9337-1428391e8aef-kubelet-config\") pod \"global-pull-secret-syncer-tlbhv\" (UID: \"51b9bc4d-f596-4c94-9337-1428391e8aef\") " pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:36:57.586414 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586268 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/51b9bc4d-f596-4c94-9337-1428391e8aef-dbus\") pod \"global-pull-secret-syncer-tlbhv\" (UID: \"51b9bc4d-f596-4c94-9337-1428391e8aef\") " pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:36:57.586414 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586288 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72d8f397-d16d-446b-9d87-c2064ba99f21-ovnkube-script-lib\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.586414 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586304 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxchk\" (UniqueName: \"kubernetes.io/projected/26965363-ec8e-46d5-8647-25aa21dd2896-kube-api-access-fxchk\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.586414 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586323 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/acaa1746-a8a5-4517-9299-9a502758b7a4-hosts-file\") pod \"node-resolver-zvt6m\" (UID: \"acaa1746-a8a5-4517-9299-9a502758b7a4\") " pod="openshift-dns/node-resolver-zvt6m" Jul 22 01:36:57.586414 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586339 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkmc6\" (UniqueName: \"kubernetes.io/projected/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-kube-api-access-fkmc6\") pod \"network-metrics-daemon-878hc\" (UID: \"8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e\") " pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:36:57.586414 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586373 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e3459ca6-711d-4b8a-b1d1-3f65d03ce297-host\") pod \"node-ca-rs2vg\" (UID: \"e3459ca6-711d-4b8a-b1d1-3f65d03ce297\") " pod="openshift-image-registry/node-ca-rs2vg" Jul 22 01:36:57.586414 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586387 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-systemd-units\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.586414 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586401 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72d8f397-d16d-446b-9d87-c2064ba99f21-ovn-node-metrics-cert\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.586414 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586401 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/acaa1746-a8a5-4517-9299-9a502758b7a4-hosts-file\") pod \"node-resolver-zvt6m\" (UID: \"acaa1746-a8a5-4517-9299-9a502758b7a4\") " pod="openshift-dns/node-resolver-zvt6m" Jul 22 01:36:57.586966 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586423 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsqkb\" (UniqueName: \"kubernetes.io/projected/886c82f1-624b-42ba-83fe-339fb2898a20-kube-api-access-tsqkb\") pod \"iptables-alerter-hz2w7\" (UID: \"886c82f1-624b-42ba-83fe-339fb2898a20\") " pod="openshift-network-operator/iptables-alerter-hz2w7" Jul 22 01:36:57.586966 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586508 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/efeab439-38f6-4ebe-8a23-0c8f6d34e743-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.586966 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586535 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-run\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.586966 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586575 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xjsp\" (UniqueName: \"kubernetes.io/projected/6c474173-c98d-4b91-88f8-8379a3bfdf82-kube-api-access-2xjsp\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.586966 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586613 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzv4z\" (UniqueName: \"kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z\") pod \"network-check-target-zrkr9\" (UID: \"4326c7c1-88b7-4ead-be08-a2a2e2497a64\") " pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:36:57.586966 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586641 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqhjh\" (UniqueName: \"kubernetes.io/projected/e3459ca6-711d-4b8a-b1d1-3f65d03ce297-kube-api-access-fqhjh\") pod \"node-ca-rs2vg\" (UID: \"e3459ca6-711d-4b8a-b1d1-3f65d03ce297\") " pod="openshift-image-registry/node-ca-rs2vg" Jul 22 01:36:57.586966 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586669 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-kubelet\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.586966 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586695 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.586966 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586723 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/efeab439-38f6-4ebe-8a23-0c8f6d34e743-cnibin\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.586966 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586744 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/efeab439-38f6-4ebe-8a23-0c8f6d34e743-os-release\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.586966 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586766 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c474173-c98d-4b91-88f8-8379a3bfdf82-tmp\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.586966 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586802 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/efeab439-38f6-4ebe-8a23-0c8f6d34e743-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.586966 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586842 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-kubelet-dir\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.586966 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586864 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-tuned\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.586966 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586886 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-run-ovn\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.586966 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586938 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-cni-bin\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.587639 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586957 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgsph\" (UniqueName: \"kubernetes.io/projected/72d8f397-d16d-446b-9d87-c2064ba99f21-kube-api-access-mgsph\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.587639 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.586986 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/efeab439-38f6-4ebe-8a23-0c8f6d34e743-cni-binary-copy\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.587639 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587001 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hxdn\" (UniqueName: \"kubernetes.io/projected/efeab439-38f6-4ebe-8a23-0c8f6d34e743-kube-api-access-2hxdn\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.587639 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587022 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgf2l\" (UniqueName: \"kubernetes.io/projected/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-kube-api-access-mgf2l\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.587639 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587054 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-host-var-lib-cni-multus\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.587639 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587072 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/886c82f1-624b-42ba-83fe-339fb2898a20-host-slash\") pod \"iptables-alerter-hz2w7\" (UID: \"886c82f1-624b-42ba-83fe-339fb2898a20\") " pod="openshift-network-operator/iptables-alerter-hz2w7" Jul 22 01:36:57.587639 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587099 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-run-systemd\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.587639 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587123 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-var-lib-openvswitch\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.587639 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587125 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/886c82f1-624b-42ba-83fe-339fb2898a20-host-slash\") pod \"iptables-alerter-hz2w7\" (UID: \"886c82f1-624b-42ba-83fe-339fb2898a20\") " pod="openshift-network-operator/iptables-alerter-hz2w7" Jul 22 01:36:57.587639 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587194 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72d8f397-d16d-446b-9d87-c2064ba99f21-env-overrides\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.587639 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587217 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-etc-selinux\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.587639 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587242 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-host-run-netns\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.587639 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587282 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-host-var-lib-kubelet\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.587639 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587305 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-sys-fs\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.587639 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587332 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-modprobe-d\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.587639 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587371 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/49ed0d1f-ebe1-4552-978c-4386cc349f09-agent-certs\") pod \"konnectivity-agent-xtb6t\" (UID: \"49ed0d1f-ebe1-4552-978c-4386cc349f09\") " pod="kube-system/konnectivity-agent-xtb6t" Jul 22 01:36:57.588091 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587395 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-etc-openvswitch\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.588091 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587415 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-host-var-lib-cni-bin\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.588091 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587447 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/26965363-ec8e-46d5-8647-25aa21dd2896-multus-daemon-config\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.588091 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587480 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-host-run-multus-certs\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.588091 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587509 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs\") pod \"network-metrics-daemon-878hc\" (UID: \"8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e\") " pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:36:57.588091 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587537 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-slash\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.588091 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587555 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-socket-dir\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.588091 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587592 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-sys\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.588091 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587619 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-lib-modules\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.588091 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587644 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-host\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.588091 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587669 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-system-cni-dir\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.588091 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587694 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acaa1746-a8a5-4517-9299-9a502758b7a4-tmp-dir\") pod \"node-resolver-zvt6m\" (UID: \"acaa1746-a8a5-4517-9299-9a502758b7a4\") " pod="openshift-dns/node-resolver-zvt6m" Jul 22 01:36:57.588091 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587725 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/49ed0d1f-ebe1-4552-978c-4386cc349f09-konnectivity-ca\") pod \"konnectivity-agent-xtb6t\" (UID: \"49ed0d1f-ebe1-4552-978c-4386cc349f09\") " pod="kube-system/konnectivity-agent-xtb6t" Jul 22 01:36:57.588091 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587751 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret\") pod \"global-pull-secret-syncer-tlbhv\" (UID: \"51b9bc4d-f596-4c94-9337-1428391e8aef\") " pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:36:57.588091 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587775 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-registration-dir\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.588091 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587795 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-sysctl-d\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.588091 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587812 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e3459ca6-711d-4b8a-b1d1-3f65d03ce297-serviceca\") pod \"node-ca-rs2vg\" (UID: \"e3459ca6-711d-4b8a-b1d1-3f65d03ce297\") " pod="openshift-image-registry/node-ca-rs2vg" Jul 22 01:36:57.588594 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587827 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-host-run-k8s-cni-cncf-io\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.588594 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587841 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-etc-kubernetes\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.588594 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587855 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-os-release\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.588594 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.587873 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-hostroot\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.589203 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.589187 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/886c82f1-624b-42ba-83fe-339fb2898a20-iptables-alerter-script\") pod \"iptables-alerter-hz2w7\" (UID: \"886c82f1-624b-42ba-83fe-339fb2898a20\") " pod="openshift-network-operator/iptables-alerter-hz2w7" Jul 22 01:36:57.589389 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.589374 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acaa1746-a8a5-4517-9299-9a502758b7a4-tmp-dir\") pod \"node-resolver-zvt6m\" (UID: \"acaa1746-a8a5-4517-9299-9a502758b7a4\") " pod="openshift-dns/node-resolver-zvt6m" Jul 22 01:36:57.594638 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:57.594613 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 01:36:57.594766 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:57.594654 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 01:36:57.594766 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:57.594670 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fzv4z for pod openshift-network-diagnostics/network-check-target-zrkr9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 01:36:57.594766 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.594654 2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 01:36:57.594862 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:57.594779 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z podName:4326c7c1-88b7-4ead-be08-a2a2e2497a64 nodeName:}" failed. No retries permitted until 2026-07-22 01:36:58.094733469 +0000 UTC m=+2.923981814 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fzv4z" (UniqueName: "kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z") pod "network-check-target-zrkr9" (UID: "4326c7c1-88b7-4ead-be08-a2a2e2497a64") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 01:36:57.598069 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.598044 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsqkb\" (UniqueName: \"kubernetes.io/projected/886c82f1-624b-42ba-83fe-339fb2898a20-kube-api-access-tsqkb\") pod \"iptables-alerter-hz2w7\" (UID: \"886c82f1-624b-42ba-83fe-339fb2898a20\") " pod="openshift-network-operator/iptables-alerter-hz2w7" Jul 22 01:36:57.598241 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.598092 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nnqz\" (UniqueName: \"kubernetes.io/projected/acaa1746-a8a5-4517-9299-9a502758b7a4-kube-api-access-8nnqz\") pod \"node-resolver-zvt6m\" (UID: \"acaa1746-a8a5-4517-9299-9a502758b7a4\") " pod="openshift-dns/node-resolver-zvt6m" Jul 22 01:36:57.603472 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.603452 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hz2w7" Jul 22 01:36:57.614407 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.614335 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 01:36:57.688567 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688532 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-systemd\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.688567 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688568 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-cnibin\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.688843 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688586 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-multus-socket-dir-parent\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.688843 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688606 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-run-ovn-kubernetes\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.688843 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688632 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/efeab439-38f6-4ebe-8a23-0c8f6d34e743-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.688843 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688654 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-sysconfig\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.688843 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688665 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-systemd\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.688843 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688672 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-multus-conf-dir\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.688843 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688708 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-multus-conf-dir\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.688843 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688733 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-run-netns\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.688843 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688748 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-multus-socket-dir-parent\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.688843 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688759 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-run-ovn-kubernetes\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.688843 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688759 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-run-openvswitch\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.688843 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688782 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-cnibin\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.688843 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688792 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-log-socket\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.688843 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688793 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-run-netns\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.688843 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688789 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-run-openvswitch\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.688843 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688801 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-sysconfig\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.688843 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688820 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-cni-netd\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.688843 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688832 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-log-socket\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.689701 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688843 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72d8f397-d16d-446b-9d87-c2064ba99f21-ovnkube-config\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.689701 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688866 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-sysctl-conf\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.689701 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688889 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-multus-cni-dir\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.689701 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688896 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-cni-netd\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.689701 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688912 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/26965363-ec8e-46d5-8647-25aa21dd2896-cni-binary-copy\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.689701 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.688979 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/51b9bc4d-f596-4c94-9337-1428391e8aef-kubelet-config\") pod \"global-pull-secret-syncer-tlbhv\" (UID: \"51b9bc4d-f596-4c94-9337-1428391e8aef\") " pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:36:57.689701 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689004 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/51b9bc4d-f596-4c94-9337-1428391e8aef-dbus\") pod \"global-pull-secret-syncer-tlbhv\" (UID: \"51b9bc4d-f596-4c94-9337-1428391e8aef\") " pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:36:57.689701 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689012 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-sysctl-conf\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.689701 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689026 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72d8f397-d16d-446b-9d87-c2064ba99f21-ovnkube-script-lib\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.689701 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689050 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxchk\" (UniqueName: \"kubernetes.io/projected/26965363-ec8e-46d5-8647-25aa21dd2896-kube-api-access-fxchk\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.689701 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689048 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/51b9bc4d-f596-4c94-9337-1428391e8aef-kubelet-config\") pod \"global-pull-secret-syncer-tlbhv\" (UID: \"51b9bc4d-f596-4c94-9337-1428391e8aef\") " pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:36:57.689701 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689075 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkmc6\" (UniqueName: \"kubernetes.io/projected/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-kube-api-access-fkmc6\") pod \"network-metrics-daemon-878hc\" (UID: \"8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e\") " pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:36:57.689701 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689077 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-multus-cni-dir\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.689701 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689100 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e3459ca6-711d-4b8a-b1d1-3f65d03ce297-host\") pod \"node-ca-rs2vg\" (UID: \"e3459ca6-711d-4b8a-b1d1-3f65d03ce297\") " pod="openshift-image-registry/node-ca-rs2vg" Jul 22 01:36:57.689701 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689124 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-systemd-units\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.689701 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689148 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72d8f397-d16d-446b-9d87-c2064ba99f21-ovn-node-metrics-cert\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.689701 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689207 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/efeab439-38f6-4ebe-8a23-0c8f6d34e743-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.690463 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689235 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-run\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.690463 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689265 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xjsp\" (UniqueName: \"kubernetes.io/projected/6c474173-c98d-4b91-88f8-8379a3bfdf82-kube-api-access-2xjsp\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.690463 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689333 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqhjh\" (UniqueName: \"kubernetes.io/projected/e3459ca6-711d-4b8a-b1d1-3f65d03ce297-kube-api-access-fqhjh\") pod \"node-ca-rs2vg\" (UID: \"e3459ca6-711d-4b8a-b1d1-3f65d03ce297\") " pod="openshift-image-registry/node-ca-rs2vg" Jul 22 01:36:57.690463 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689371 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/efeab439-38f6-4ebe-8a23-0c8f6d34e743-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.690463 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689436 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-kubelet\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.690463 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689524 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/72d8f397-d16d-446b-9d87-c2064ba99f21-ovnkube-config\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.690463 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689579 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e3459ca6-711d-4b8a-b1d1-3f65d03ce297-host\") pod \"node-ca-rs2vg\" (UID: \"e3459ca6-711d-4b8a-b1d1-3f65d03ce297\") " pod="openshift-image-registry/node-ca-rs2vg" Jul 22 01:36:57.690463 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689705 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/72d8f397-d16d-446b-9d87-c2064ba99f21-ovnkube-script-lib\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.690463 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689750 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/26965363-ec8e-46d5-8647-25aa21dd2896-cni-binary-copy\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.690463 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689378 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-kubelet\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.690463 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689802 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/51b9bc4d-f596-4c94-9337-1428391e8aef-dbus\") pod \"global-pull-secret-syncer-tlbhv\" (UID: \"51b9bc4d-f596-4c94-9337-1428391e8aef\") " pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:36:57.690463 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689817 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.690463 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689820 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-run\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.690463 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689848 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-systemd-units\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.690463 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689845 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/efeab439-38f6-4ebe-8a23-0c8f6d34e743-cnibin\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.690463 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689876 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.690463 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689889 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/efeab439-38f6-4ebe-8a23-0c8f6d34e743-os-release\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.691255 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689919 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c474173-c98d-4b91-88f8-8379a3bfdf82-tmp\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.691255 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689922 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/efeab439-38f6-4ebe-8a23-0c8f6d34e743-cnibin\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.691255 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689944 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/efeab439-38f6-4ebe-8a23-0c8f6d34e743-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.691255 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689973 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/efeab439-38f6-4ebe-8a23-0c8f6d34e743-os-release\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.691255 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.689970 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-kubelet-dir\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.691255 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690006 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-tuned\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.691255 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690026 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-run-ovn\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.691255 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690037 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-kubelet-dir\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.691255 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690053 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-cni-bin\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.691255 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690078 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgsph\" (UniqueName: \"kubernetes.io/projected/72d8f397-d16d-446b-9d87-c2064ba99f21-kube-api-access-mgsph\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.691255 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690084 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-run-ovn\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.691255 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690102 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/efeab439-38f6-4ebe-8a23-0c8f6d34e743-cni-binary-copy\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.691255 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690119 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-cni-bin\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.691255 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690126 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hxdn\" (UniqueName: \"kubernetes.io/projected/efeab439-38f6-4ebe-8a23-0c8f6d34e743-kube-api-access-2hxdn\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.691255 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690152 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgf2l\" (UniqueName: \"kubernetes.io/projected/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-kube-api-access-mgf2l\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.691255 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690176 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-host-var-lib-cni-multus\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.691255 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690223 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-run-systemd\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.692092 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690245 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-var-lib-openvswitch\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.692092 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690253 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/efeab439-38f6-4ebe-8a23-0c8f6d34e743-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.692092 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690269 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72d8f397-d16d-446b-9d87-c2064ba99f21-env-overrides\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.692092 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690302 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-etc-selinux\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.692092 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690328 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-run-systemd\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.692092 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690382 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-host-run-netns\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.692092 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690119 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/efeab439-38f6-4ebe-8a23-0c8f6d34e743-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.692092 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690502 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-host-var-lib-cni-multus\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.692092 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690514 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-etc-selinux\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.692092 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690331 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-host-run-netns\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.692092 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690548 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/efeab439-38f6-4ebe-8a23-0c8f6d34e743-cni-binary-copy\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.692092 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690554 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-var-lib-openvswitch\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.692092 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690557 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-host-var-lib-kubelet\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.692092 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690588 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-host-var-lib-kubelet\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.692092 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690623 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-sys-fs\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.692092 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690661 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-modprobe-d\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.692092 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690670 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-sys-fs\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.692903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690689 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/49ed0d1f-ebe1-4552-978c-4386cc349f09-agent-certs\") pod \"konnectivity-agent-xtb6t\" (UID: \"49ed0d1f-ebe1-4552-978c-4386cc349f09\") " pod="kube-system/konnectivity-agent-xtb6t" Jul 22 01:36:57.692903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690694 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/72d8f397-d16d-446b-9d87-c2064ba99f21-env-overrides\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.692903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690713 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-etc-openvswitch\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.692903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690756 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-host-var-lib-cni-bin\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.692903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690780 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/26965363-ec8e-46d5-8647-25aa21dd2896-multus-daemon-config\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.692903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690781 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-modprobe-d\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.692903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690804 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-host-run-multus-certs\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.692903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690829 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs\") pod \"network-metrics-daemon-878hc\" (UID: \"8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e\") " pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:36:57.692903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690834 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-host-var-lib-cni-bin\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.692903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690851 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-slash\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.692903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690878 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-etc-openvswitch\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.692903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690874 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-socket-dir\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.692903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690914 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-sys\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.692903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690938 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-lib-modules\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.692903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690967 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-host\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.692903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690991 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-system-cni-dir\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.692903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691015 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/49ed0d1f-ebe1-4552-978c-4386cc349f09-konnectivity-ca\") pod \"konnectivity-agent-xtb6t\" (UID: \"49ed0d1f-ebe1-4552-978c-4386cc349f09\") " pod="kube-system/konnectivity-agent-xtb6t" Jul 22 01:36:57.692903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691039 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret\") pod \"global-pull-secret-syncer-tlbhv\" (UID: \"51b9bc4d-f596-4c94-9337-1428391e8aef\") " pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:36:57.693439 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:57.691045 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 01:36:57.693439 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691051 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-sys\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.693439 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691062 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-registration-dir\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.693439 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691087 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-sysctl-d\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.693439 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691100 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-host-run-multus-certs\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.693439 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:57.691116 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs podName:8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e nodeName:}" failed. No retries permitted until 2026-07-22 01:36:58.191094327 +0000 UTC m=+3.020342673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs") pod "network-metrics-daemon-878hc" (UID: "8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 01:36:57.693439 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:57.691178 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 01:36:57.693439 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691289 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-registration-dir\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.693439 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:57.691400 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret podName:51b9bc4d-f596-4c94-9337-1428391e8aef nodeName:}" failed. No retries permitted until 2026-07-22 01:36:58.191375785 +0000 UTC m=+3.020624117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret") pod "global-pull-secret-syncer-tlbhv" (UID: "51b9bc4d-f596-4c94-9337-1428391e8aef") : object "kube-system"/"original-pull-secret" not registered Jul 22 01:36:57.693439 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691422 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e3459ca6-711d-4b8a-b1d1-3f65d03ce297-serviceca\") pod \"node-ca-rs2vg\" (UID: \"e3459ca6-711d-4b8a-b1d1-3f65d03ce297\") " pod="openshift-image-registry/node-ca-rs2vg" Jul 22 01:36:57.693439 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691423 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/26965363-ec8e-46d5-8647-25aa21dd2896-multus-daemon-config\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.693439 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691468 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-host-run-k8s-cni-cncf-io\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.693439 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691497 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-etc-kubernetes\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.693439 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691521 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-os-release\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.693439 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691547 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-hostroot\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.693439 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691571 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-node-log\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.693439 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691577 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-sysctl-d\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.694050 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691594 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-device-dir\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.694050 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691622 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-var-lib-kubelet\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.694050 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691624 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-etc-kubernetes\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.694050 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691643 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/49ed0d1f-ebe1-4552-978c-4386cc349f09-konnectivity-ca\") pod \"konnectivity-agent-xtb6t\" (UID: \"49ed0d1f-ebe1-4552-978c-4386cc349f09\") " pod="kube-system/konnectivity-agent-xtb6t" Jul 22 01:36:57.694050 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691696 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-os-release\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.694050 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691735 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-hostroot\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.694050 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691751 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-lib-modules\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.694050 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691771 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-node-log\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.694050 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691805 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-host\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.694050 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.690966 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/72d8f397-d16d-446b-9d87-c2064ba99f21-host-slash\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.694050 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691816 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-device-dir\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.694050 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691856 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-var-lib-kubelet\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.694050 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691880 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-system-cni-dir\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.694050 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691897 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/26965363-ec8e-46d5-8647-25aa21dd2896-host-run-k8s-cni-cncf-io\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.694050 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.691952 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/efeab439-38f6-4ebe-8a23-0c8f6d34e743-system-cni-dir\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.694050 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.692021 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/efeab439-38f6-4ebe-8a23-0c8f6d34e743-system-cni-dir\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.694050 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.692050 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-kubernetes\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.694050 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.692068 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e3459ca6-711d-4b8a-b1d1-3f65d03ce297-serviceca\") pod \"node-ca-rs2vg\" (UID: \"e3459ca6-711d-4b8a-b1d1-3f65d03ce297\") " pod="openshift-image-registry/node-ca-rs2vg" Jul 22 01:36:57.694568 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.692141 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-kubernetes\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.694568 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.692215 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-socket-dir\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.694568 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.692638 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c474173-c98d-4b91-88f8-8379a3bfdf82-tmp\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.694568 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.692739 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/72d8f397-d16d-446b-9d87-c2064ba99f21-ovn-node-metrics-cert\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.694568 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.692879 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6c474173-c98d-4b91-88f8-8379a3bfdf82-etc-tuned\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.694771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.694755 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/49ed0d1f-ebe1-4552-978c-4386cc349f09-agent-certs\") pod \"konnectivity-agent-xtb6t\" (UID: \"49ed0d1f-ebe1-4552-978c-4386cc349f09\") " pod="kube-system/konnectivity-agent-xtb6t" Jul 22 01:36:57.698251 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.698227 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxchk\" (UniqueName: \"kubernetes.io/projected/26965363-ec8e-46d5-8647-25aa21dd2896-kube-api-access-fxchk\") pod \"multus-phv9v\" (UID: \"26965363-ec8e-46d5-8647-25aa21dd2896\") " pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.702294 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.702269 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgf2l\" (UniqueName: \"kubernetes.io/projected/5c71b2f1-cb20-45d0-b328-76b4fed1fa36-kube-api-access-mgf2l\") pod \"aws-ebs-csi-driver-node-skllr\" (UID: \"5c71b2f1-cb20-45d0-b328-76b4fed1fa36\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.703096 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.703068 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xjsp\" (UniqueName: \"kubernetes.io/projected/6c474173-c98d-4b91-88f8-8379a3bfdf82-kube-api-access-2xjsp\") pod \"tuned-blbsx\" (UID: \"6c474173-c98d-4b91-88f8-8379a3bfdf82\") " pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.703715 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.703690 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkmc6\" (UniqueName: \"kubernetes.io/projected/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-kube-api-access-fkmc6\") pod \"network-metrics-daemon-878hc\" (UID: \"8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e\") " pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:36:57.703715 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.703700 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqhjh\" (UniqueName: \"kubernetes.io/projected/e3459ca6-711d-4b8a-b1d1-3f65d03ce297-kube-api-access-fqhjh\") pod \"node-ca-rs2vg\" (UID: \"e3459ca6-711d-4b8a-b1d1-3f65d03ce297\") " pod="openshift-image-registry/node-ca-rs2vg" Jul 22 01:36:57.703977 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.703947 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hxdn\" (UniqueName: \"kubernetes.io/projected/efeab439-38f6-4ebe-8a23-0c8f6d34e743-kube-api-access-2hxdn\") pod \"multus-additional-cni-plugins-9dqvw\" (UID: \"efeab439-38f6-4ebe-8a23-0c8f6d34e743\") " pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.704551 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.704536 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgsph\" (UniqueName: \"kubernetes.io/projected/72d8f397-d16d-446b-9d87-c2064ba99f21-kube-api-access-mgsph\") pod \"ovnkube-node-7gfnj\" (UID: \"72d8f397-d16d-446b-9d87-c2064ba99f21\") " pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:57.716075 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.716056 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-blbsx" Jul 22 01:36:57.753824 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:57.753777 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c474173_c98d_4b91_88f8_8379a3bfdf82.slice/crio-2de0cd0ed094e35ab65df79154021d76f575288612948030d56f99678ebc5f6a WatchSource:0}: Error finding container 2de0cd0ed094e35ab65df79154021d76f575288612948030d56f99678ebc5f6a: Status 404 returned error can't find the container with id 2de0cd0ed094e35ab65df79154021d76f575288612948030d56f99678ebc5f6a Jul 22 01:36:57.755234 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:57.755197 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae3d302a83e4b95708739b07d02ee6e8.slice/crio-e41d1c5dd67bb9b497b0fbade8291160fd26c07dbf472abb3d2629bbd7f96a0f WatchSource:0}: Error finding container e41d1c5dd67bb9b497b0fbade8291160fd26c07dbf472abb3d2629bbd7f96a0f: Status 404 returned error can't find the container with id e41d1c5dd67bb9b497b0fbade8291160fd26c07dbf472abb3d2629bbd7f96a0f Jul 22 01:36:57.755989 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:57.755973 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f63d47aab8d4135421bfb4b85518502.slice/crio-9bb0ac3483afa36b0e404961a55bb2438fabe5575a159b464390e01c02f9bc88 WatchSource:0}: Error finding container 9bb0ac3483afa36b0e404961a55bb2438fabe5575a159b464390e01c02f9bc88: Status 404 returned error can't find the container with id 9bb0ac3483afa36b0e404961a55bb2438fabe5575a159b464390e01c02f9bc88 Jul 22 01:36:57.758674 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.758645 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 01:36:57.871995 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.871888 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zvt6m" Jul 22 01:36:57.881465 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:57.881435 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacaa1746_a8a5_4517_9299_9a502758b7a4.slice/crio-110ba332d8b3b955a278978b7fc3c335a8542199adc95123f0dfa4c43efa0d65 WatchSource:0}: Error finding container 110ba332d8b3b955a278978b7fc3c335a8542199adc95123f0dfa4c43efa0d65: Status 404 returned error can't find the container with id 110ba332d8b3b955a278978b7fc3c335a8542199adc95123f0dfa4c43efa0d65 Jul 22 01:36:57.911943 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.911893 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9dqvw" Jul 22 01:36:57.920974 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:57.920944 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefeab439_38f6_4ebe_8a23_0c8f6d34e743.slice/crio-a0c4c3bf935d64f1c00ddd550c47af02d3086a233849a78d3aa04605c2811c13 WatchSource:0}: Error finding container a0c4c3bf935d64f1c00ddd550c47af02d3086a233849a78d3aa04605c2811c13: Status 404 returned error can't find the container with id a0c4c3bf935d64f1c00ddd550c47af02d3086a233849a78d3aa04605c2811c13 Jul 22 01:36:57.927812 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.927788 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-phv9v" Jul 22 01:36:57.941624 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:57.941602 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26965363_ec8e_46d5_8647_25aa21dd2896.slice/crio-b0e2138997d03781fd7c465fef8d2c9ac7af406930b331c9f387fce9aef5bc56 WatchSource:0}: Error finding container b0e2138997d03781fd7c465fef8d2c9ac7af406930b331c9f387fce9aef5bc56: Status 404 returned error can't find the container with id b0e2138997d03781fd7c465fef8d2c9ac7af406930b331c9f387fce9aef5bc56 Jul 22 01:36:57.945038 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.945017 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rs2vg" Jul 22 01:36:57.952182 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:57.952158 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod886c82f1_624b_42ba_83fe_339fb2898a20.slice/crio-fada8f8bbfe28d288bc1880482b62a077ad8443359bc01a56524635c525a7b40 WatchSource:0}: Error finding container fada8f8bbfe28d288bc1880482b62a077ad8443359bc01a56524635c525a7b40: Status 404 returned error can't find the container with id fada8f8bbfe28d288bc1880482b62a077ad8443359bc01a56524635c525a7b40 Jul 22 01:36:57.953904 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:57.953884 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3459ca6_711d_4b8a_b1d1_3f65d03ce297.slice/crio-0621bc5a4142c5d71618d1cef944a74527794503632174998fd906331266996c WatchSource:0}: Error finding container 0621bc5a4142c5d71618d1cef944a74527794503632174998fd906331266996c: Status 404 returned error can't find the container with id 0621bc5a4142c5d71618d1cef944a74527794503632174998fd906331266996c Jul 22 01:36:57.959326 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.959302 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" Jul 22 01:36:57.968656 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:57.968626 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c71b2f1_cb20_45d0_b328_76b4fed1fa36.slice/crio-c162382b20b47f98bd196b2390bb7c3b5730cb4c3ac9a44b259439f71f5c8e89 WatchSource:0}: Error finding container c162382b20b47f98bd196b2390bb7c3b5730cb4c3ac9a44b259439f71f5c8e89: Status 404 returned error can't find the container with id c162382b20b47f98bd196b2390bb7c3b5730cb4c3ac9a44b259439f71f5c8e89 Jul 22 01:36:57.975045 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.975024 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xtb6t" Jul 22 01:36:57.984297 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:57.984268 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49ed0d1f_ebe1_4552_978c_4386cc349f09.slice/crio-3ae552107279d2292290506ae33e1462bbaf8fd56de0712147f43599714514cc WatchSource:0}: Error finding container 3ae552107279d2292290506ae33e1462bbaf8fd56de0712147f43599714514cc: Status 404 returned error can't find the container with id 3ae552107279d2292290506ae33e1462bbaf8fd56de0712147f43599714514cc Jul 22 01:36:57.994458 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:57.994433 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:36:58.003181 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:36:58.003155 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72d8f397_d16d_446b_9d87_c2064ba99f21.slice/crio-544ec4df44eb03f2f922f58dd14e7f3ab6a878b0204e98ffda3e0c03da546889 WatchSource:0}: Error finding container 544ec4df44eb03f2f922f58dd14e7f3ab6a878b0204e98ffda3e0c03da546889: Status 404 returned error can't find the container with id 544ec4df44eb03f2f922f58dd14e7f3ab6a878b0204e98ffda3e0c03da546889 Jul 22 01:36:58.094969 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:58.094932 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzv4z\" (UniqueName: \"kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z\") pod \"network-check-target-zrkr9\" (UID: \"4326c7c1-88b7-4ead-be08-a2a2e2497a64\") " pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:36:58.095156 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:58.095114 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 01:36:58.095156 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:58.095139 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 01:36:58.095156 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:58.095150 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fzv4z for pod openshift-network-diagnostics/network-check-target-zrkr9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 01:36:58.095265 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:58.095201 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z podName:4326c7c1-88b7-4ead-be08-a2a2e2497a64 nodeName:}" failed. No retries permitted until 2026-07-22 01:36:59.095187937 +0000 UTC m=+3.924436265 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzv4z" (UniqueName: "kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z") pod "network-check-target-zrkr9" (UID: "4326c7c1-88b7-4ead-be08-a2a2e2497a64") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 01:36:58.195830 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:58.195732 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs\") pod \"network-metrics-daemon-878hc\" (UID: \"8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e\") " pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:36:58.195830 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:58.195774 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret\") pod \"global-pull-secret-syncer-tlbhv\" (UID: \"51b9bc4d-f596-4c94-9337-1428391e8aef\") " pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:36:58.196011 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:58.195871 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 01:36:58.196011 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:58.195874 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 01:36:58.196011 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:58.195922 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret podName:51b9bc4d-f596-4c94-9337-1428391e8aef nodeName:}" failed. No retries permitted until 2026-07-22 01:36:59.195909346 +0000 UTC m=+4.025157676 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret") pod "global-pull-secret-syncer-tlbhv" (UID: "51b9bc4d-f596-4c94-9337-1428391e8aef") : object "kube-system"/"original-pull-secret" not registered Jul 22 01:36:58.196011 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:58.195938 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs podName:8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e nodeName:}" failed. No retries permitted until 2026-07-22 01:36:59.195930108 +0000 UTC m=+4.025178438 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs") pod "network-metrics-daemon-878hc" (UID: "8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 01:36:58.493328 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:58.493250 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 01:36:58.531846 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:58.531775 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 01:31:57 +0000 UTC" deadline="2028-03-30 23:46:47.79170286 +0000 UTC" Jul 22 01:36:58.531846 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:58.531844 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14830h9m49.25986325s" Jul 22 01:36:58.620416 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:58.620383 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:36:58.621330 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:58.620555 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-878hc" podUID="8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e" Jul 22 01:36:58.702530 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:58.702423 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-80.ec2.internal" event={"ID":"ae3d302a83e4b95708739b07d02ee6e8","Type":"ContainerStarted","Data":"e41d1c5dd67bb9b497b0fbade8291160fd26c07dbf472abb3d2629bbd7f96a0f"} Jul 22 01:36:58.706169 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:58.706085 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-blbsx" event={"ID":"6c474173-c98d-4b91-88f8-8379a3bfdf82","Type":"ContainerStarted","Data":"2de0cd0ed094e35ab65df79154021d76f575288612948030d56f99678ebc5f6a"} Jul 22 01:36:58.710054 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:58.709930 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" event={"ID":"72d8f397-d16d-446b-9d87-c2064ba99f21","Type":"ContainerStarted","Data":"544ec4df44eb03f2f922f58dd14e7f3ab6a878b0204e98ffda3e0c03da546889"} Jul 22 01:36:58.720563 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:58.718490 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" event={"ID":"5c71b2f1-cb20-45d0-b328-76b4fed1fa36","Type":"ContainerStarted","Data":"c162382b20b47f98bd196b2390bb7c3b5730cb4c3ac9a44b259439f71f5c8e89"} Jul 22 01:36:58.720563 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:58.720403 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-phv9v" event={"ID":"26965363-ec8e-46d5-8647-25aa21dd2896","Type":"ContainerStarted","Data":"b0e2138997d03781fd7c465fef8d2c9ac7af406930b331c9f387fce9aef5bc56"} Jul 22 01:36:58.735629 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:58.735589 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9dqvw" event={"ID":"efeab439-38f6-4ebe-8a23-0c8f6d34e743","Type":"ContainerStarted","Data":"a0c4c3bf935d64f1c00ddd550c47af02d3086a233849a78d3aa04605c2811c13"} Jul 22 01:36:58.752255 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:58.751985 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal" event={"ID":"9f63d47aab8d4135421bfb4b85518502","Type":"ContainerStarted","Data":"9bb0ac3483afa36b0e404961a55bb2438fabe5575a159b464390e01c02f9bc88"} Jul 22 01:36:58.759583 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:58.759546 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xtb6t" event={"ID":"49ed0d1f-ebe1-4552-978c-4386cc349f09","Type":"ContainerStarted","Data":"3ae552107279d2292290506ae33e1462bbaf8fd56de0712147f43599714514cc"} Jul 22 01:36:58.774798 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:58.774641 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rs2vg" event={"ID":"e3459ca6-711d-4b8a-b1d1-3f65d03ce297","Type":"ContainerStarted","Data":"0621bc5a4142c5d71618d1cef944a74527794503632174998fd906331266996c"} Jul 22 01:36:58.788240 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:58.788124 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hz2w7" event={"ID":"886c82f1-624b-42ba-83fe-339fb2898a20","Type":"ContainerStarted","Data":"fada8f8bbfe28d288bc1880482b62a077ad8443359bc01a56524635c525a7b40"} Jul 22 01:36:58.793685 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:58.793514 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zvt6m" event={"ID":"acaa1746-a8a5-4517-9299-9a502758b7a4","Type":"ContainerStarted","Data":"110ba332d8b3b955a278978b7fc3c335a8542199adc95123f0dfa4c43efa0d65"} Jul 22 01:36:59.102692 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:59.102600 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzv4z\" (UniqueName: \"kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z\") pod \"network-check-target-zrkr9\" (UID: \"4326c7c1-88b7-4ead-be08-a2a2e2497a64\") " pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:36:59.102864 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:59.102773 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 01:36:59.102864 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:59.102792 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 01:36:59.102864 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:59.102805 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fzv4z for pod openshift-network-diagnostics/network-check-target-zrkr9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 01:36:59.102864 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:59.102863 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z podName:4326c7c1-88b7-4ead-be08-a2a2e2497a64 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:01.102845289 +0000 UTC m=+5.932093627 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzv4z" (UniqueName: "kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z") pod "network-check-target-zrkr9" (UID: "4326c7c1-88b7-4ead-be08-a2a2e2497a64") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 01:36:59.203003 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:59.202965 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs\") pod \"network-metrics-daemon-878hc\" (UID: \"8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e\") " pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:36:59.203188 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:59.203014 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret\") pod \"global-pull-secret-syncer-tlbhv\" (UID: \"51b9bc4d-f596-4c94-9337-1428391e8aef\") " pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:36:59.203188 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:59.203131 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 01:36:59.203328 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:59.203192 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret podName:51b9bc4d-f596-4c94-9337-1428391e8aef nodeName:}" failed. No retries permitted until 2026-07-22 01:37:01.203175596 +0000 UTC m=+6.032423935 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret") pod "global-pull-secret-syncer-tlbhv" (UID: "51b9bc4d-f596-4c94-9337-1428391e8aef") : object "kube-system"/"original-pull-secret" not registered Jul 22 01:36:59.203692 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:59.203664 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 01:36:59.203900 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:59.203720 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs podName:8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e nodeName:}" failed. No retries permitted until 2026-07-22 01:37:01.203706004 +0000 UTC m=+6.032954345 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs") pod "network-metrics-daemon-878hc" (UID: "8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 01:36:59.535176 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:59.533000 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 01:31:57 +0000 UTC" deadline="2028-04-04 04:14:15.784155316 +0000 UTC" Jul 22 01:36:59.535176 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:59.533052 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14930h37m16.251108002s" Jul 22 01:36:59.622122 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:59.621993 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:36:59.622122 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:59.622119 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrkr9" podUID="4326c7c1-88b7-4ead-be08-a2a2e2497a64" Jul 22 01:36:59.622706 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:36:59.622527 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:36:59.622706 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:36:59.622659 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbhv" podUID="51b9bc4d-f596-4c94-9337-1428391e8aef" Jul 22 01:37:00.623365 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:00.620356 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:00.623365 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:00.620532 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-878hc" podUID="8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e" Jul 22 01:37:01.120817 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:01.120145 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzv4z\" (UniqueName: \"kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z\") pod \"network-check-target-zrkr9\" (UID: \"4326c7c1-88b7-4ead-be08-a2a2e2497a64\") " pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:01.120817 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:01.120412 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 01:37:01.120817 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:01.120436 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 01:37:01.120817 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:01.120460 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fzv4z for pod openshift-network-diagnostics/network-check-target-zrkr9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 01:37:01.120817 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:01.120517 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z podName:4326c7c1-88b7-4ead-be08-a2a2e2497a64 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:05.120499418 +0000 UTC m=+9.949747746 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzv4z" (UniqueName: "kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z") pod "network-check-target-zrkr9" (UID: "4326c7c1-88b7-4ead-be08-a2a2e2497a64") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 01:37:01.221208 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:01.221170 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs\") pod \"network-metrics-daemon-878hc\" (UID: \"8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e\") " pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:01.221412 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:01.221225 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret\") pod \"global-pull-secret-syncer-tlbhv\" (UID: \"51b9bc4d-f596-4c94-9337-1428391e8aef\") " pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:01.221412 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:01.221366 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 01:37:01.221534 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:01.221426 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret podName:51b9bc4d-f596-4c94-9337-1428391e8aef nodeName:}" failed. No retries permitted until 2026-07-22 01:37:05.221407221 +0000 UTC m=+10.050655553 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret") pod "global-pull-secret-syncer-tlbhv" (UID: "51b9bc4d-f596-4c94-9337-1428391e8aef") : object "kube-system"/"original-pull-secret" not registered Jul 22 01:37:01.221854 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:01.221834 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 01:37:01.221938 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:01.221892 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs podName:8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e nodeName:}" failed. No retries permitted until 2026-07-22 01:37:05.221876685 +0000 UTC m=+10.051125015 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs") pod "network-metrics-daemon-878hc" (UID: "8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 01:37:01.621674 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:01.621648 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:01.621855 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:01.621795 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrkr9" podUID="4326c7c1-88b7-4ead-be08-a2a2e2497a64" Jul 22 01:37:01.621926 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:01.621648 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:01.621998 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:01.621976 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbhv" podUID="51b9bc4d-f596-4c94-9337-1428391e8aef" Jul 22 01:37:02.620403 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:02.620368 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:02.620864 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:02.620535 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-878hc" podUID="8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e" Jul 22 01:37:03.620296 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:03.620260 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:03.620506 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:03.620420 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrkr9" podUID="4326c7c1-88b7-4ead-be08-a2a2e2497a64" Jul 22 01:37:03.620876 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:03.620824 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:03.620928 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:03.620906 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbhv" podUID="51b9bc4d-f596-4c94-9337-1428391e8aef" Jul 22 01:37:04.620241 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:04.619972 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:04.620241 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:04.620119 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-878hc" podUID="8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e" Jul 22 01:37:05.163709 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:05.163018 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzv4z\" (UniqueName: \"kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z\") pod \"network-check-target-zrkr9\" (UID: \"4326c7c1-88b7-4ead-be08-a2a2e2497a64\") " pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:05.163709 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:05.163228 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 01:37:05.163709 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:05.163252 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 01:37:05.163709 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:05.163265 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fzv4z for pod openshift-network-diagnostics/network-check-target-zrkr9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 01:37:05.163709 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:05.163361 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z podName:4326c7c1-88b7-4ead-be08-a2a2e2497a64 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:13.163326643 +0000 UTC m=+17.992574977 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzv4z" (UniqueName: "kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z") pod "network-check-target-zrkr9" (UID: "4326c7c1-88b7-4ead-be08-a2a2e2497a64") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 01:37:05.263504 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:05.263457 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs\") pod \"network-metrics-daemon-878hc\" (UID: \"8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e\") " pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:05.263688 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:05.263516 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret\") pod \"global-pull-secret-syncer-tlbhv\" (UID: \"51b9bc4d-f596-4c94-9337-1428391e8aef\") " pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:05.263688 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:05.263619 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 01:37:05.263805 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:05.263703 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs podName:8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e nodeName:}" failed. No retries permitted until 2026-07-22 01:37:13.263679773 +0000 UTC m=+18.092928114 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs") pod "network-metrics-daemon-878hc" (UID: "8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 01:37:05.264113 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:05.263626 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 01:37:05.264113 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:05.264083 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret podName:51b9bc4d-f596-4c94-9337-1428391e8aef nodeName:}" failed. No retries permitted until 2026-07-22 01:37:13.264066045 +0000 UTC m=+18.093314375 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret") pod "global-pull-secret-syncer-tlbhv" (UID: "51b9bc4d-f596-4c94-9337-1428391e8aef") : object "kube-system"/"original-pull-secret" not registered Jul 22 01:37:05.624415 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:05.623710 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:05.624415 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:05.623833 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrkr9" podUID="4326c7c1-88b7-4ead-be08-a2a2e2497a64" Jul 22 01:37:05.624415 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:05.624206 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:05.624415 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:05.624308 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbhv" podUID="51b9bc4d-f596-4c94-9337-1428391e8aef" Jul 22 01:37:06.620127 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:06.620088 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:06.620592 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:06.620245 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-878hc" podUID="8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e" Jul 22 01:37:07.620484 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:07.620328 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:07.620960 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:07.620476 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbhv" podUID="51b9bc4d-f596-4c94-9337-1428391e8aef" Jul 22 01:37:07.620960 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:07.620332 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:07.620960 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:07.620928 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrkr9" podUID="4326c7c1-88b7-4ead-be08-a2a2e2497a64" Jul 22 01:37:08.620252 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:08.620218 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:08.620467 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:08.620361 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-878hc" podUID="8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e" Jul 22 01:37:09.619579 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:09.619528 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:09.620003 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:09.619700 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbhv" podUID="51b9bc4d-f596-4c94-9337-1428391e8aef" Jul 22 01:37:09.620003 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:09.619758 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:09.620003 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:09.619863 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrkr9" podUID="4326c7c1-88b7-4ead-be08-a2a2e2497a64" Jul 22 01:37:10.619701 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:10.619665 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:10.620110 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:10.619796 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-878hc" podUID="8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e" Jul 22 01:37:11.620597 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:11.620562 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:11.621077 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:11.620562 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:11.621077 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:11.620692 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbhv" podUID="51b9bc4d-f596-4c94-9337-1428391e8aef" Jul 22 01:37:11.621077 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:11.620734 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrkr9" podUID="4326c7c1-88b7-4ead-be08-a2a2e2497a64" Jul 22 01:37:12.619459 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:12.619418 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:12.619640 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:12.619551 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-878hc" podUID="8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e" Jul 22 01:37:13.223430 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:13.223390 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzv4z\" (UniqueName: \"kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z\") pod \"network-check-target-zrkr9\" (UID: \"4326c7c1-88b7-4ead-be08-a2a2e2497a64\") " pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:13.223852 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:13.223574 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 01:37:13.223852 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:13.223598 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 01:37:13.223852 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:13.223610 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fzv4z for pod openshift-network-diagnostics/network-check-target-zrkr9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 01:37:13.223852 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:13.223666 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z podName:4326c7c1-88b7-4ead-be08-a2a2e2497a64 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:29.223652344 +0000 UTC m=+34.052900673 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzv4z" (UniqueName: "kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z") pod "network-check-target-zrkr9" (UID: "4326c7c1-88b7-4ead-be08-a2a2e2497a64") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 01:37:13.324152 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:13.324111 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs\") pod \"network-metrics-daemon-878hc\" (UID: \"8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e\") " pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:13.324380 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:13.324162 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret\") pod \"global-pull-secret-syncer-tlbhv\" (UID: \"51b9bc4d-f596-4c94-9337-1428391e8aef\") " pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:13.324380 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:13.324296 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 01:37:13.324380 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:13.324299 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 01:37:13.324550 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:13.324383 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret podName:51b9bc4d-f596-4c94-9337-1428391e8aef nodeName:}" failed. No retries permitted until 2026-07-22 01:37:29.324364429 +0000 UTC m=+34.153612774 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret") pod "global-pull-secret-syncer-tlbhv" (UID: "51b9bc4d-f596-4c94-9337-1428391e8aef") : object "kube-system"/"original-pull-secret" not registered Jul 22 01:37:13.324550 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:13.324403 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs podName:8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e nodeName:}" failed. No retries permitted until 2026-07-22 01:37:29.324393726 +0000 UTC m=+34.153642051 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs") pod "network-metrics-daemon-878hc" (UID: "8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 01:37:13.620043 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:13.619952 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:13.620189 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:13.620072 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrkr9" podUID="4326c7c1-88b7-4ead-be08-a2a2e2497a64" Jul 22 01:37:13.620189 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:13.620127 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:13.620266 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:13.620243 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbhv" podUID="51b9bc4d-f596-4c94-9337-1428391e8aef" Jul 22 01:37:14.620104 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:14.620014 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:14.620567 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:14.620156 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-878hc" podUID="8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e" Jul 22 01:37:15.621064 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:15.621020 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:15.621064 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:15.621054 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:15.621536 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:15.621144 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbhv" podUID="51b9bc4d-f596-4c94-9337-1428391e8aef" Jul 22 01:37:15.621536 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:15.621248 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrkr9" podUID="4326c7c1-88b7-4ead-be08-a2a2e2497a64" Jul 22 01:37:16.620167 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:16.620126 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:16.620387 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:16.620281 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-878hc" podUID="8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e" Jul 22 01:37:17.619968 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:17.619927 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:17.620467 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:17.620073 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbhv" podUID="51b9bc4d-f596-4c94-9337-1428391e8aef" Jul 22 01:37:17.620467 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:17.620135 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:17.620467 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:17.620251 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrkr9" podUID="4326c7c1-88b7-4ead-be08-a2a2e2497a64" Jul 22 01:37:18.620258 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:18.619851 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:18.620746 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:18.620415 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-878hc" podUID="8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e" Jul 22 01:37:18.847255 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:18.847209 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-phv9v" event={"ID":"26965363-ec8e-46d5-8647-25aa21dd2896","Type":"ContainerStarted","Data":"14e4cf5bbe8fc05c3b141d366fe75b72516156a73edca0444eee4074a4f69750"} Jul 22 01:37:18.849288 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:18.849256 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-80.ec2.internal" event={"ID":"ae3d302a83e4b95708739b07d02ee6e8","Type":"ContainerStarted","Data":"a62de6565a8f5d891135df3c347d8329f5e2fcf565339647a3f716b30385cb7e"} Jul 22 01:37:18.851168 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:18.851144 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-blbsx" event={"ID":"6c474173-c98d-4b91-88f8-8379a3bfdf82","Type":"ContainerStarted","Data":"c2cba1a735f3a4fe62dc98ba8d2d2d3f60c3f799a5509f5d72adf380478f24e7"} Jul 22 01:37:18.854373 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:18.854337 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 01:37:18.854753 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:18.854731 2559 generic.go:356] "Generic (PLEG): container finished" podID="72d8f397-d16d-446b-9d87-c2064ba99f21" containerID="23fd08727c43cb751e696119d25fd0c528538f7403815bbc6157f37a17cd62f9" exitCode=1 Jul 22 01:37:18.854862 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:18.854766 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" event={"ID":"72d8f397-d16d-446b-9d87-c2064ba99f21","Type":"ContainerStarted","Data":"9175a2af3fdb9146fd4ed57b748c4c2e321eedddb15a7a62f5bb9f11e0ff31c2"} Jul 22 01:37:18.854921 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:18.854860 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" event={"ID":"72d8f397-d16d-446b-9d87-c2064ba99f21","Type":"ContainerStarted","Data":"004aa7a6aae2170e50246405e86da32592ec11031bab3e82afd278cc5ff0ac1d"} Jul 22 01:37:18.854921 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:18.854891 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" event={"ID":"72d8f397-d16d-446b-9d87-c2064ba99f21","Type":"ContainerStarted","Data":"abd27953c4a974057ad8128290581b240ed5d1228a5829b22e6c317d68f8fe6b"} Jul 22 01:37:18.854921 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:18.854919 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" event={"ID":"72d8f397-d16d-446b-9d87-c2064ba99f21","Type":"ContainerStarted","Data":"404312fc170726753b0c6aa00d11e6d18ecc8be8ec9041d8c41b6f01da2edcec"} Jul 22 01:37:18.855042 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:18.854946 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" event={"ID":"72d8f397-d16d-446b-9d87-c2064ba99f21","Type":"ContainerDied","Data":"23fd08727c43cb751e696119d25fd0c528538f7403815bbc6157f37a17cd62f9"} Jul 22 01:37:18.855042 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:18.854974 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" event={"ID":"72d8f397-d16d-446b-9d87-c2064ba99f21","Type":"ContainerStarted","Data":"2bdff39bb13bf942716754b17bab55427ac5a1811d7ad68687a2deadfc69d52d"} Jul 22 01:37:18.866482 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:18.866418 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-phv9v" podStartSLOduration=3.647483377 podStartE2EDuration="23.866398755s" totalImagesPullingTime="20.218915378s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:36:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:18.866138905 +0000 UTC m=+23.695387254" watchObservedRunningTime="2026-07-22 01:37:18.866398755 +0000 UTC m=+23.695647106" Jul 22 01:37:18.898701 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:18.898598 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-blbsx" podStartSLOduration=2.846633289 podStartE2EDuration="22.898580379s" totalImagesPullingTime="20.05194709s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:18.884561909 +0000 UTC m=+23.713810261" watchObservedRunningTime="2026-07-22 01:37:18.898580379 +0000 UTC m=+23.727828846" Jul 22 01:37:18.898988 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:18.898969 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-80.ec2.internal" podStartSLOduration=21.898962318 podStartE2EDuration="21.898962318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:36:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:18.898370813 +0000 UTC m=+23.727619153" watchObservedRunningTime="2026-07-22 01:37:18.898962318 +0000 UTC m=+23.728210666" Jul 22 01:37:19.619626 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:19.619584 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:19.619823 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:19.619636 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:19.619823 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:19.619734 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbhv" podUID="51b9bc4d-f596-4c94-9337-1428391e8aef" Jul 22 01:37:19.619929 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:19.619862 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrkr9" podUID="4326c7c1-88b7-4ead-be08-a2a2e2497a64" Jul 22 01:37:19.859198 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:19.859138 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xtb6t" event={"ID":"49ed0d1f-ebe1-4552-978c-4386cc349f09","Type":"ContainerStarted","Data":"527f780b3c8caddfff523a8ebc0c993d9361d9f7ed339144e73f1bffb80b35b6"} Jul 22 01:37:19.860983 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:19.860946 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rs2vg" event={"ID":"e3459ca6-711d-4b8a-b1d1-3f65d03ce297","Type":"ContainerStarted","Data":"b7c451cc213f8264c93886d31b185e7a7db42c91dcbbedc3f22f1c48934de50e"} Jul 22 01:37:19.862525 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:19.862502 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hz2w7" event={"ID":"886c82f1-624b-42ba-83fe-339fb2898a20","Type":"ContainerStarted","Data":"8b78495e001712c5adfee1f46ce2061464932000e03b047fa5bbb7379d1a6b61"} Jul 22 01:37:19.864108 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:19.864075 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zvt6m" event={"ID":"acaa1746-a8a5-4517-9299-9a502758b7a4","Type":"ContainerStarted","Data":"25499c89da5386d07977185eb9463f1de11e4ee86a897e3ca8b875a275376d33"} Jul 22 01:37:19.865702 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:19.865677 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" event={"ID":"5c71b2f1-cb20-45d0-b328-76b4fed1fa36","Type":"ContainerStarted","Data":"bd664c4b290031fa8ce2f23c2bcd8cb482e6d5c52b2f55192f9ad857db65ec2f"} Jul 22 01:37:19.867236 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:19.867201 2559 generic.go:356] "Generic (PLEG): container finished" podID="efeab439-38f6-4ebe-8a23-0c8f6d34e743" containerID="e07ea13de2df35671e19deca367a75a5c02310b168e81b250414d6e745c1885d" exitCode=0 Jul 22 01:37:19.867355 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:19.867278 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9dqvw" event={"ID":"efeab439-38f6-4ebe-8a23-0c8f6d34e743","Type":"ContainerDied","Data":"e07ea13de2df35671e19deca367a75a5c02310b168e81b250414d6e745c1885d"} Jul 22 01:37:19.869073 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:19.869047 2559 generic.go:356] "Generic (PLEG): container finished" podID="9f63d47aab8d4135421bfb4b85518502" containerID="6fcc493f623478e2cccb65a5e0c0113327361aafb3a5b1973519bf92a4cfa18b" exitCode=0 Jul 22 01:37:19.869156 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:19.869135 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal" event={"ID":"9f63d47aab8d4135421bfb4b85518502","Type":"ContainerDied","Data":"6fcc493f623478e2cccb65a5e0c0113327361aafb3a5b1973519bf92a4cfa18b"} Jul 22 01:37:19.891827 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:19.891766 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xtb6t" podStartSLOduration=5.068570651 podStartE2EDuration="24.891743806s" totalImagesPullingTime="19.823173155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:36:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:19.874703146 +0000 UTC m=+24.703951496" watchObservedRunningTime="2026-07-22 01:37:19.891743806 +0000 UTC m=+24.720992215" Jul 22 01:37:19.906458 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:19.906401 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zvt6m" podStartSLOduration=4.980192947 podStartE2EDuration="24.906384275s" totalImagesPullingTime="19.926191328s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:36:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:19.905744495 +0000 UTC m=+24.734992844" watchObservedRunningTime="2026-07-22 01:37:19.906384275 +0000 UTC m=+24.735632622" Jul 22 01:37:19.941233 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:19.941175 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hz2w7" podStartSLOduration=5.079338271 podStartE2EDuration="24.941158021s" totalImagesPullingTime="19.86181975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:36:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:19.941063123 +0000 UTC m=+24.770311471" watchObservedRunningTime="2026-07-22 01:37:19.941158021 +0000 UTC m=+24.770406368" Jul 22 01:37:19.955239 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:19.955180 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rs2vg" podStartSLOduration=5.101340463 podStartE2EDuration="24.955165856s" totalImagesPullingTime="19.853825393s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:36:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:19.954627659 +0000 UTC m=+24.783876006" watchObservedRunningTime="2026-07-22 01:37:19.955165856 +0000 UTC m=+24.784414260" Jul 22 01:37:20.349668 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:20.349501 2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 01:37:20.477644 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:20.477555 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xtb6t" Jul 22 01:37:20.478210 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:20.478192 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xtb6t" Jul 22 01:37:20.554121 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:20.554011 2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T01:37:20.349668175Z","UUID":"fa4e9264-dcdb-4ed1-8450-cb06049b0f36","Handler":null,"Name":"","Endpoint":""} Jul 22 01:37:20.556069 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:20.556043 2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 01:37:20.556202 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:20.556089 2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 01:37:20.619584 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:20.619552 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:20.619758 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:20.619702 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-878hc" podUID="8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e" Jul 22 01:37:20.874408 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:20.874367 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" event={"ID":"5c71b2f1-cb20-45d0-b328-76b4fed1fa36","Type":"ContainerStarted","Data":"d9056d6280557beb1150c463e630eabe24376fc7aaf6db45bd72c20288c9a138"} Jul 22 01:37:20.877330 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:20.877295 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal" event={"ID":"9f63d47aab8d4135421bfb4b85518502","Type":"ContainerStarted","Data":"849bc524a0080190465f99d185f3f99e14b3540de4d34ebb856663d2987542b3"} Jul 22 01:37:20.894507 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:20.894456 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-80.ec2.internal" podStartSLOduration=23.894436942 podStartE2EDuration="23.894436942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:36:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:20.89418536 +0000 UTC m=+25.723433708" watchObservedRunningTime="2026-07-22 01:37:20.894436942 +0000 UTC m=+25.723685293" Jul 22 01:37:21.123256 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:21.123204 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xtb6t" Jul 22 01:37:21.123881 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:21.123830 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xtb6t" Jul 22 01:37:21.620430 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:21.620402 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:21.620579 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:21.620414 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:21.620579 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:21.620540 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbhv" podUID="51b9bc4d-f596-4c94-9337-1428391e8aef" Jul 22 01:37:21.620579 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:21.620566 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrkr9" podUID="4326c7c1-88b7-4ead-be08-a2a2e2497a64" Jul 22 01:37:21.882603 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:21.882525 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 01:37:21.883054 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:21.882977 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" event={"ID":"72d8f397-d16d-446b-9d87-c2064ba99f21","Type":"ContainerStarted","Data":"5882e8d7c90468af0d14bbce6a04ccbe6f4996e8f1ef616f834458b6b6ffde83"} Jul 22 01:37:21.885329 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:21.885300 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" event={"ID":"5c71b2f1-cb20-45d0-b328-76b4fed1fa36","Type":"ContainerStarted","Data":"4933b6d6572585ad829e04d4949e513dfdfd0e8bc664c555f35b2ed0330bea71"} Jul 22 01:37:21.907105 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:21.907044 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-skllr" podStartSLOduration=4.671987734 podStartE2EDuration="26.907026286s" totalImagesPullingTime="22.235038552s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:36:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:21.90657413 +0000 UTC m=+26.735822479" watchObservedRunningTime="2026-07-22 01:37:21.907026286 +0000 UTC m=+26.736274634" Jul 22 01:37:22.619638 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:22.619600 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:22.619839 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:22.619745 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-878hc" podUID="8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e" Jul 22 01:37:23.620042 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:23.620002 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:23.620709 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:23.620002 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:23.620709 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:23.620127 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrkr9" podUID="4326c7c1-88b7-4ead-be08-a2a2e2497a64" Jul 22 01:37:23.620709 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:23.620306 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbhv" podUID="51b9bc4d-f596-4c94-9337-1428391e8aef" Jul 22 01:37:23.893882 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:23.893800 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 01:37:23.894219 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:23.894196 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" event={"ID":"72d8f397-d16d-446b-9d87-c2064ba99f21","Type":"ContainerStarted","Data":"bcc99f67aef005267563e361857fb56ffd06a1f96644d6ca484cf27378c86853"} Jul 22 01:37:23.894639 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:23.894483 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:37:23.894749 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:23.894723 2559 scope.go:117] "RemoveContainer" containerID="23fd08727c43cb751e696119d25fd0c528538f7403815bbc6157f37a17cd62f9" Jul 22 01:37:23.912588 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:23.912556 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:37:24.619913 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:24.619877 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:24.620062 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:24.619994 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-878hc" podUID="8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e" Jul 22 01:37:25.380705 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:25.380506 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zrkr9"] Jul 22 01:37:25.380856 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:25.380772 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:25.380894 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:25.380852 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrkr9" podUID="4326c7c1-88b7-4ead-be08-a2a2e2497a64" Jul 22 01:37:25.383253 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:25.383225 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-878hc"] Jul 22 01:37:25.383406 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:25.383365 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:25.383509 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:25.383484 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-878hc" podUID="8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e" Jul 22 01:37:25.394318 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:25.394204 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tlbhv"] Jul 22 01:37:25.394318 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:25.394302 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:25.394579 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:25.394408 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbhv" podUID="51b9bc4d-f596-4c94-9337-1428391e8aef" Jul 22 01:37:25.900729 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:25.900695 2559 generic.go:356] "Generic (PLEG): container finished" podID="efeab439-38f6-4ebe-8a23-0c8f6d34e743" containerID="1b203910d69e6cd720fb7bbf5ea367476166a53bc38697b4ea721c8aa6040ec7" exitCode=0 Jul 22 01:37:25.901297 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:25.900799 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9dqvw" event={"ID":"efeab439-38f6-4ebe-8a23-0c8f6d34e743","Type":"ContainerDied","Data":"1b203910d69e6cd720fb7bbf5ea367476166a53bc38697b4ea721c8aa6040ec7"} Jul 22 01:37:25.904605 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:25.904586 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 01:37:25.905019 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:25.904993 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" event={"ID":"72d8f397-d16d-446b-9d87-c2064ba99f21","Type":"ContainerStarted","Data":"ca4e4422accbe9f763b41ef4e8d31685518d69d6c0a6fd572e02fe5422c8019c"} Jul 22 01:37:25.905396 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:25.905377 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:37:25.905478 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:25.905452 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:37:25.921279 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:25.921252 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:37:25.949253 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:25.949200 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" podStartSLOduration=10.123787161 podStartE2EDuration="29.949184842s" totalImagesPullingTime="19.825397681s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:25.948899627 +0000 UTC m=+30.778147974" watchObservedRunningTime="2026-07-22 01:37:25.949184842 +0000 UTC m=+30.778433180" Jul 22 01:37:26.619483 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:26.619457 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:26.619579 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:26.619559 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrkr9" podUID="4326c7c1-88b7-4ead-be08-a2a2e2497a64" Jul 22 01:37:26.908869 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:26.908776 2559 generic.go:356] "Generic (PLEG): container finished" podID="efeab439-38f6-4ebe-8a23-0c8f6d34e743" containerID="da16adc4ae0ab81d51e84d1276d5867a310d6f339314c767182c1f32be13567a" exitCode=0 Jul 22 01:37:26.909220 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:26.908863 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9dqvw" event={"ID":"efeab439-38f6-4ebe-8a23-0c8f6d34e743","Type":"ContainerDied","Data":"da16adc4ae0ab81d51e84d1276d5867a310d6f339314c767182c1f32be13567a"} Jul 22 01:37:27.620143 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:27.620119 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:27.620226 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:27.620125 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:27.620294 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:27.620272 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-878hc" podUID="8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e" Jul 22 01:37:27.620378 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:27.620295 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbhv" podUID="51b9bc4d-f596-4c94-9337-1428391e8aef" Jul 22 01:37:27.913157 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:27.913123 2559 generic.go:356] "Generic (PLEG): container finished" podID="efeab439-38f6-4ebe-8a23-0c8f6d34e743" containerID="5c6a87fb31730abf696f24bda4c33a0f856dc98a7c146914d0f3e0f9513ef977" exitCode=0 Jul 22 01:37:27.913561 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:27.913186 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9dqvw" event={"ID":"efeab439-38f6-4ebe-8a23-0c8f6d34e743","Type":"ContainerDied","Data":"5c6a87fb31730abf696f24bda4c33a0f856dc98a7c146914d0f3e0f9513ef977"} Jul 22 01:37:28.619777 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:28.619746 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:28.619936 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:28.619862 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrkr9" podUID="4326c7c1-88b7-4ead-be08-a2a2e2497a64" Jul 22 01:37:29.245102 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:29.245048 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzv4z\" (UniqueName: \"kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z\") pod \"network-check-target-zrkr9\" (UID: \"4326c7c1-88b7-4ead-be08-a2a2e2497a64\") " pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:29.245599 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:29.245194 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 01:37:29.245599 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:29.245215 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 01:37:29.245599 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:29.245228 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fzv4z for pod openshift-network-diagnostics/network-check-target-zrkr9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 01:37:29.245599 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:29.245290 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z podName:4326c7c1-88b7-4ead-be08-a2a2e2497a64 nodeName:}" failed. No retries permitted until 2026-07-22 01:38:01.245275561 +0000 UTC m=+66.074523888 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fzv4z" (UniqueName: "kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z") pod "network-check-target-zrkr9" (UID: "4326c7c1-88b7-4ead-be08-a2a2e2497a64") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 01:37:29.345972 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:29.345936 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs\") pod \"network-metrics-daemon-878hc\" (UID: \"8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e\") " pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:29.345972 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:29.345977 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret\") pod \"global-pull-secret-syncer-tlbhv\" (UID: \"51b9bc4d-f596-4c94-9337-1428391e8aef\") " pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:29.346185 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:29.346097 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 01:37:29.346185 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:29.346169 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret podName:51b9bc4d-f596-4c94-9337-1428391e8aef nodeName:}" failed. No retries permitted until 2026-07-22 01:38:01.346150199 +0000 UTC m=+66.175398539 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret") pod "global-pull-secret-syncer-tlbhv" (UID: "51b9bc4d-f596-4c94-9337-1428391e8aef") : object "kube-system"/"original-pull-secret" not registered Jul 22 01:37:29.346292 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:29.346097 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 01:37:29.346292 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:29.346273 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs podName:8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e nodeName:}" failed. No retries permitted until 2026-07-22 01:38:01.346250598 +0000 UTC m=+66.175498925 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs") pod "network-metrics-daemon-878hc" (UID: "8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 01:37:29.619996 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:29.619786 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:29.620168 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:29.619786 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:29.620168 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:29.620114 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-878hc" podUID="8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e" Jul 22 01:37:29.620293 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:29.620164 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tlbhv" podUID="51b9bc4d-f596-4c94-9337-1428391e8aef" Jul 22 01:37:30.619725 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:30.619694 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:30.620167 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:30.619798 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zrkr9" podUID="4326c7c1-88b7-4ead-be08-a2a2e2497a64" Jul 22 01:37:31.025524 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.025435 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-80.ec2.internal" event="NodeReady" Jul 22 01:37:31.025666 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.025592 2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 01:37:31.063814 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.063781 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-687855c694-q2wpf"] Jul 22 01:37:31.067890 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.067867 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.070696 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.070670 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fnkgz\"" Jul 22 01:37:31.070844 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.070670 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 22 01:37:31.070844 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.070750 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 22 01:37:31.071759 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.071737 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 22 01:37:31.077763 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.077734 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 22 01:37:31.083659 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.083631 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-687855c694-q2wpf"] Jul 22 01:37:31.085472 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.085444 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-g2zdj"] Jul 22 01:37:31.089188 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.089162 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qhr55"] Jul 22 01:37:31.089361 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.089300 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g2zdj" Jul 22 01:37:31.092856 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.092708 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 01:37:31.092856 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.092728 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 01:37:31.092856 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.092708 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fhl28\"" Jul 22 01:37:31.093106 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.092937 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qhr55" Jul 22 01:37:31.095198 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.095176 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 01:37:31.095505 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.095486 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 01:37:31.095634 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.095485 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 01:37:31.095634 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.095549 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-v8ltf\"" Jul 22 01:37:31.098955 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.098920 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qhr55"] Jul 22 01:37:31.101283 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.101251 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g2zdj"] Jul 22 01:37:31.160592 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.160559 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-image-registry-private-configuration\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.160765 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.160597 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzx75\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-kube-api-access-xzx75\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.160765 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.160630 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.160765 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.160716 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-bound-sa-token\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.160765 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.160759 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-trusted-ca\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.160986 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.160800 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-installation-pull-secrets\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.160986 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.160895 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-ca-trust-extracted\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.160986 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.160935 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-certificates\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.261853 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.261808 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb80e90e-d908-490b-a9da-bb1d624d6615-tmp-dir\") pod \"dns-default-g2zdj\" (UID: \"cb80e90e-d908-490b-a9da-bb1d624d6615\") " pod="openshift-dns/dns-default-g2zdj" Jul 22 01:37:31.261853 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.261857 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls\") pod \"dns-default-g2zdj\" (UID: \"cb80e90e-d908-490b-a9da-bb1d624d6615\") " pod="openshift-dns/dns-default-g2zdj" Jul 22 01:37:31.262100 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.261902 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hxf8\" (UniqueName: \"kubernetes.io/projected/0df0a233-5828-4368-96ca-134ab7641aed-kube-api-access-7hxf8\") pod \"ingress-canary-qhr55\" (UID: \"0df0a233-5828-4368-96ca-134ab7641aed\") " pod="openshift-ingress-canary/ingress-canary-qhr55" Jul 22 01:37:31.262100 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.261935 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-ca-trust-extracted\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.262100 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.261965 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-certificates\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.262257 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.262103 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb80e90e-d908-490b-a9da-bb1d624d6615-config-volume\") pod \"dns-default-g2zdj\" (UID: \"cb80e90e-d908-490b-a9da-bb1d624d6615\") " pod="openshift-dns/dns-default-g2zdj" Jul 22 01:37:31.262257 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.262137 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-image-registry-private-configuration\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.262257 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.262177 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bqrw\" (UniqueName: \"kubernetes.io/projected/cb80e90e-d908-490b-a9da-bb1d624d6615-kube-api-access-5bqrw\") pod \"dns-default-g2zdj\" (UID: \"cb80e90e-d908-490b-a9da-bb1d624d6615\") " pod="openshift-dns/dns-default-g2zdj" Jul 22 01:37:31.262257 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.262229 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzx75\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-kube-api-access-xzx75\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.262449 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.262259 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.262449 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.262312 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-bound-sa-token\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.262449 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:31.262367 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 01:37:31.262449 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.262371 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-trusted-ca\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.262449 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:31.262383 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-687855c694-q2wpf: secret "image-registry-tls" not found Jul 22 01:37:31.262449 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.262384 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-ca-trust-extracted\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.262449 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.262409 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-installation-pull-secrets\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.262449 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:31.262442 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls podName:8ea58868-1c24-48f4-8c3d-2427d0a6dbe2 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:31.762421186 +0000 UTC m=+36.591669526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls") pod "image-registry-687855c694-q2wpf" (UID: "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2") : secret "image-registry-tls" not found Jul 22 01:37:31.262703 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.262460 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0df0a233-5828-4368-96ca-134ab7641aed-cert\") pod \"ingress-canary-qhr55\" (UID: \"0df0a233-5828-4368-96ca-134ab7641aed\") " pod="openshift-ingress-canary/ingress-canary-qhr55" Jul 22 01:37:31.262703 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.262641 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-certificates\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.263261 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.263236 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-trusted-ca\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.267074 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.267048 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-image-registry-private-configuration\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.267074 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.267048 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-installation-pull-secrets\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.273335 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.273303 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-bound-sa-token\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.273816 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.273761 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzx75\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-kube-api-access-xzx75\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.364374 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.364213 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb80e90e-d908-490b-a9da-bb1d624d6615-config-volume\") pod \"dns-default-g2zdj\" (UID: \"cb80e90e-d908-490b-a9da-bb1d624d6615\") " pod="openshift-dns/dns-default-g2zdj" Jul 22 01:37:31.364374 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.364284 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bqrw\" (UniqueName: \"kubernetes.io/projected/cb80e90e-d908-490b-a9da-bb1d624d6615-kube-api-access-5bqrw\") pod \"dns-default-g2zdj\" (UID: \"cb80e90e-d908-490b-a9da-bb1d624d6615\") " pod="openshift-dns/dns-default-g2zdj" Jul 22 01:37:31.364374 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.364375 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0df0a233-5828-4368-96ca-134ab7641aed-cert\") pod \"ingress-canary-qhr55\" (UID: \"0df0a233-5828-4368-96ca-134ab7641aed\") " pod="openshift-ingress-canary/ingress-canary-qhr55" Jul 22 01:37:31.364666 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.364413 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb80e90e-d908-490b-a9da-bb1d624d6615-tmp-dir\") pod \"dns-default-g2zdj\" (UID: \"cb80e90e-d908-490b-a9da-bb1d624d6615\") " pod="openshift-dns/dns-default-g2zdj" Jul 22 01:37:31.364666 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.364446 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls\") pod \"dns-default-g2zdj\" (UID: \"cb80e90e-d908-490b-a9da-bb1d624d6615\") " pod="openshift-dns/dns-default-g2zdj" Jul 22 01:37:31.364666 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.364490 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hxf8\" (UniqueName: \"kubernetes.io/projected/0df0a233-5828-4368-96ca-134ab7641aed-kube-api-access-7hxf8\") pod \"ingress-canary-qhr55\" (UID: \"0df0a233-5828-4368-96ca-134ab7641aed\") " pod="openshift-ingress-canary/ingress-canary-qhr55" Jul 22 01:37:31.365508 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.365472 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb80e90e-d908-490b-a9da-bb1d624d6615-config-volume\") pod \"dns-default-g2zdj\" (UID: \"cb80e90e-d908-490b-a9da-bb1d624d6615\") " pod="openshift-dns/dns-default-g2zdj" Jul 22 01:37:31.367021 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:31.366994 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 01:37:31.367504 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:31.367486 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls podName:cb80e90e-d908-490b-a9da-bb1d624d6615 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:31.867457886 +0000 UTC m=+36.696706217 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls") pod "dns-default-g2zdj" (UID: "cb80e90e-d908-490b-a9da-bb1d624d6615") : secret "dns-default-metrics-tls" not found Jul 22 01:37:31.367605 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.367546 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb80e90e-d908-490b-a9da-bb1d624d6615-tmp-dir\") pod \"dns-default-g2zdj\" (UID: \"cb80e90e-d908-490b-a9da-bb1d624d6615\") " pod="openshift-dns/dns-default-g2zdj" Jul 22 01:37:31.369206 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.369184 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0df0a233-5828-4368-96ca-134ab7641aed-cert\") pod \"ingress-canary-qhr55\" (UID: \"0df0a233-5828-4368-96ca-134ab7641aed\") " pod="openshift-ingress-canary/ingress-canary-qhr55" Jul 22 01:37:31.374185 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.374132 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hxf8\" (UniqueName: \"kubernetes.io/projected/0df0a233-5828-4368-96ca-134ab7641aed-kube-api-access-7hxf8\") pod \"ingress-canary-qhr55\" (UID: \"0df0a233-5828-4368-96ca-134ab7641aed\") " pod="openshift-ingress-canary/ingress-canary-qhr55" Jul 22 01:37:31.374521 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.374493 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bqrw\" (UniqueName: \"kubernetes.io/projected/cb80e90e-d908-490b-a9da-bb1d624d6615-kube-api-access-5bqrw\") pod \"dns-default-g2zdj\" (UID: \"cb80e90e-d908-490b-a9da-bb1d624d6615\") " pod="openshift-dns/dns-default-g2zdj" Jul 22 01:37:31.410856 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.410819 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qhr55" Jul 22 01:37:31.619739 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.619644 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:37:31.620321 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.619644 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:37:31.622751 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.622729 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 01:37:31.622874 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.622863 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fbrqb\"" Jul 22 01:37:31.622966 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.622946 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 01:37:31.767872 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.767829 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:31.768068 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:31.768024 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 01:37:31.768068 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:31.768051 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-687855c694-q2wpf: secret "image-registry-tls" not found Jul 22 01:37:31.768185 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:31.768127 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls podName:8ea58868-1c24-48f4-8c3d-2427d0a6dbe2 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:32.76810341 +0000 UTC m=+37.597351740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls") pod "image-registry-687855c694-q2wpf" (UID: "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2") : secret "image-registry-tls" not found Jul 22 01:37:31.869131 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:31.869073 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls\") pod \"dns-default-g2zdj\" (UID: \"cb80e90e-d908-490b-a9da-bb1d624d6615\") " pod="openshift-dns/dns-default-g2zdj" Jul 22 01:37:31.869324 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:31.869253 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 01:37:31.869403 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:31.869332 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls podName:cb80e90e-d908-490b-a9da-bb1d624d6615 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:32.869311812 +0000 UTC m=+37.698560151 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls") pod "dns-default-g2zdj" (UID: "cb80e90e-d908-490b-a9da-bb1d624d6615") : secret "dns-default-metrics-tls" not found Jul 22 01:37:32.619717 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:32.619687 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:37:32.622821 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:32.622774 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 01:37:32.623424 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:32.622891 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tqzqp\"" Jul 22 01:37:32.623698 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:32.623681 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 01:37:32.666625 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:32.666593 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qhr55"] Jul 22 01:37:32.712113 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:37:32.711929 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0df0a233_5828_4368_96ca_134ab7641aed.slice/crio-3b894e630ae83a132831d7d386ec18f10411a20c4db233e9320166ff90239f7f WatchSource:0}: Error finding container 3b894e630ae83a132831d7d386ec18f10411a20c4db233e9320166ff90239f7f: Status 404 returned error can't find the container with id 3b894e630ae83a132831d7d386ec18f10411a20c4db233e9320166ff90239f7f Jul 22 01:37:32.778040 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:32.777380 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:32.778040 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:32.777532 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 01:37:32.778040 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:32.777548 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-687855c694-q2wpf: secret "image-registry-tls" not found Jul 22 01:37:32.778040 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:32.777609 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls podName:8ea58868-1c24-48f4-8c3d-2427d0a6dbe2 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:34.777589192 +0000 UTC m=+39.606837517 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls") pod "image-registry-687855c694-q2wpf" (UID: "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2") : secret "image-registry-tls" not found Jul 22 01:37:32.878088 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:32.878053 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls\") pod \"dns-default-g2zdj\" (UID: \"cb80e90e-d908-490b-a9da-bb1d624d6615\") " pod="openshift-dns/dns-default-g2zdj" Jul 22 01:37:32.878321 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:32.878205 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 01:37:32.878321 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:32.878287 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls podName:cb80e90e-d908-490b-a9da-bb1d624d6615 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:34.878263305 +0000 UTC m=+39.707511630 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls") pod "dns-default-g2zdj" (UID: "cb80e90e-d908-490b-a9da-bb1d624d6615") : secret "dns-default-metrics-tls" not found Jul 22 01:37:32.927794 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:32.927757 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9dqvw" event={"ID":"efeab439-38f6-4ebe-8a23-0c8f6d34e743","Type":"ContainerStarted","Data":"bcbdfab87f2865fcd93e1537c9c7dfbfd75b65372cf0aa1a7e2aab796c9c87ff"} Jul 22 01:37:32.928873 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:32.928853 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qhr55" event={"ID":"0df0a233-5828-4368-96ca-134ab7641aed","Type":"ContainerStarted","Data":"3b894e630ae83a132831d7d386ec18f10411a20c4db233e9320166ff90239f7f"} Jul 22 01:37:33.933798 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:33.933763 2559 generic.go:356] "Generic (PLEG): container finished" podID="efeab439-38f6-4ebe-8a23-0c8f6d34e743" containerID="bcbdfab87f2865fcd93e1537c9c7dfbfd75b65372cf0aa1a7e2aab796c9c87ff" exitCode=0 Jul 22 01:37:33.934168 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:33.933806 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9dqvw" event={"ID":"efeab439-38f6-4ebe-8a23-0c8f6d34e743","Type":"ContainerDied","Data":"bcbdfab87f2865fcd93e1537c9c7dfbfd75b65372cf0aa1a7e2aab796c9c87ff"} Jul 22 01:37:34.794281 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:34.794218 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:34.794519 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:34.794410 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 01:37:34.794519 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:34.794434 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-687855c694-q2wpf: secret "image-registry-tls" not found Jul 22 01:37:34.794519 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:34.794509 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls podName:8ea58868-1c24-48f4-8c3d-2427d0a6dbe2 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:38.794487872 +0000 UTC m=+43.623736201 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls") pod "image-registry-687855c694-q2wpf" (UID: "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2") : secret "image-registry-tls" not found Jul 22 01:37:34.895141 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:34.895069 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls\") pod \"dns-default-g2zdj\" (UID: \"cb80e90e-d908-490b-a9da-bb1d624d6615\") " pod="openshift-dns/dns-default-g2zdj" Jul 22 01:37:34.895340 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:34.895266 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 01:37:34.895507 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:34.895360 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls podName:cb80e90e-d908-490b-a9da-bb1d624d6615 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:38.895325159 +0000 UTC m=+43.724573487 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls") pod "dns-default-g2zdj" (UID: "cb80e90e-d908-490b-a9da-bb1d624d6615") : secret "dns-default-metrics-tls" not found Jul 22 01:37:34.939751 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:34.939708 2559 generic.go:356] "Generic (PLEG): container finished" podID="efeab439-38f6-4ebe-8a23-0c8f6d34e743" containerID="b2705dbbcc994cf009b470297d1c88253668b2ba95f4a626cfee10f70ea7375a" exitCode=0 Jul 22 01:37:34.940223 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:34.939758 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9dqvw" event={"ID":"efeab439-38f6-4ebe-8a23-0c8f6d34e743","Type":"ContainerDied","Data":"b2705dbbcc994cf009b470297d1c88253668b2ba95f4a626cfee10f70ea7375a"} Jul 22 01:37:35.946267 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:35.946225 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9dqvw" event={"ID":"efeab439-38f6-4ebe-8a23-0c8f6d34e743","Type":"ContainerStarted","Data":"e7ff105b892850132ed84c76b66830f79478644f4e2f25906aed2b5517576ac1"} Jul 22 01:37:35.970209 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:35.970156 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9dqvw" podStartSLOduration=9.596677543 podStartE2EDuration="40.970139344s" totalImagesPullingTime="31.373461801s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:36:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:35.968545582 +0000 UTC m=+40.797793931" watchObservedRunningTime="2026-07-22 01:37:35.970139344 +0000 UTC m=+40.799387691" Jul 22 01:37:36.950836 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:36.950592 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qhr55" event={"ID":"0df0a233-5828-4368-96ca-134ab7641aed","Type":"ContainerStarted","Data":"86c46c8490345fe2f396056063933e5ddab45345b840a6120698c979fd0824da"} Jul 22 01:37:36.968681 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:36.968323 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qhr55" podStartSLOduration=1.9918249399999999 podStartE2EDuration="5.968299256s" totalImagesPullingTime="3.976474316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:37:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:36.966941393 +0000 UTC m=+41.796189743" watchObservedRunningTime="2026-07-22 01:37:36.968299256 +0000 UTC m=+41.797547603" Jul 22 01:37:38.826885 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:38.826837 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:38.827273 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:38.826961 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 01:37:38.827273 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:38.826976 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-687855c694-q2wpf: secret "image-registry-tls" not found Jul 22 01:37:38.827273 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:38.827038 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls podName:8ea58868-1c24-48f4-8c3d-2427d0a6dbe2 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:46.827021338 +0000 UTC m=+51.656269684 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls") pod "image-registry-687855c694-q2wpf" (UID: "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2") : secret "image-registry-tls" not found Jul 22 01:37:38.927681 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:38.927636 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls\") pod \"dns-default-g2zdj\" (UID: \"cb80e90e-d908-490b-a9da-bb1d624d6615\") " pod="openshift-dns/dns-default-g2zdj" Jul 22 01:37:38.927851 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:38.927793 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 01:37:38.927891 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:38.927864 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls podName:cb80e90e-d908-490b-a9da-bb1d624d6615 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:46.927846933 +0000 UTC m=+51.757095259 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls") pod "dns-default-g2zdj" (UID: "cb80e90e-d908-490b-a9da-bb1d624d6615") : secret "dns-default-metrics-tls" not found Jul 22 01:37:40.281363 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.281307 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-68jls"] Jul 22 01:37:40.317003 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.316959 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-flhhx"] Jul 22 01:37:40.317169 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.317100 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-68jls" Jul 22 01:37:40.319567 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.319535 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 22 01:37:40.319720 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.319597 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-xzlmf\"" Jul 22 01:37:40.319720 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.319548 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 22 01:37:40.330640 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.330570 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6jd8v"] Jul 22 01:37:40.330871 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.330719 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.333294 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.333272 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 22 01:37:40.333445 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.333407 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-2q87d\"" Jul 22 01:37:40.333445 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.333426 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 01:37:40.333445 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.333407 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 22 01:37:40.333569 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.333410 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 01:37:40.340801 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.340723 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 22 01:37:40.346707 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.346677 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb"] Jul 22 01:37:40.346839 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.346782 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6jd8v" Jul 22 01:37:40.348863 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.348824 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 22 01:37:40.349056 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.349042 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 22 01:37:40.349157 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.349096 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 22 01:37:40.349290 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.349188 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 22 01:37:40.349290 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.349247 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-d6wjw\"" Jul 22 01:37:40.368810 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.368773 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-68jls"] Jul 22 01:37:40.368993 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.368847 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb"] Jul 22 01:37:40.368993 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.368881 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-flhhx"] Jul 22 01:37:40.368993 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.368899 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6jd8v"] Jul 22 01:37:40.368993 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.368905 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb" Jul 22 01:37:40.371041 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.371020 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 22 01:37:40.371471 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.371292 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ljfv8\"" Jul 22 01:37:40.371471 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.371376 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 22 01:37:40.371633 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.371597 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 22 01:37:40.372490 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.372467 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-frfzq"] Jul 22 01:37:40.386901 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.386840 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw"] Jul 22 01:37:40.387041 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.387019 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-frfzq" Jul 22 01:37:40.389506 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.389482 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-jh24n\"" Jul 22 01:37:40.397166 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.397141 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wjp2"] Jul 22 01:37:40.397315 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.397265 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" Jul 22 01:37:40.400328 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.400302 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 22 01:37:40.400526 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.400303 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 01:37:40.400603 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.400564 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6jxxw\"" Jul 22 01:37:40.400731 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.400712 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 22 01:37:40.400818 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.400797 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 01:37:40.415359 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.415311 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-z6wbm"] Jul 22 01:37:40.415501 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.415444 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wjp2" Jul 22 01:37:40.417530 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.417504 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-2sbxj\"" Jul 22 01:37:40.417890 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.417871 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 22 01:37:40.417890 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.417871 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 22 01:37:40.418040 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.417938 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 22 01:37:40.418040 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.417982 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 22 01:37:40.438716 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.438681 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f7b6603-7aa5-411a-8956-40e0982d98f4-service-ca-bundle\") pod \"insights-operator-555d9d7885-flhhx\" (UID: \"2f7b6603-7aa5-411a-8956-40e0982d98f4\") " pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.438716 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.438722 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bhbr\" (UniqueName: \"kubernetes.io/projected/2f7b6603-7aa5-411a-8956-40e0982d98f4-kube-api-access-5bhbr\") pod \"insights-operator-555d9d7885-flhhx\" (UID: \"2f7b6603-7aa5-411a-8956-40e0982d98f4\") " pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.439153 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.438741 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f7b6603-7aa5-411a-8956-40e0982d98f4-tmp\") pod \"insights-operator-555d9d7885-flhhx\" (UID: \"2f7b6603-7aa5-411a-8956-40e0982d98f4\") " pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.439153 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.438759 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2f7b6603-7aa5-411a-8956-40e0982d98f4-snapshots\") pod \"insights-operator-555d9d7885-flhhx\" (UID: \"2f7b6603-7aa5-411a-8956-40e0982d98f4\") " pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.439153 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.438813 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f721d6eb-f694-4e62-9133-12edd4f6c81c-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-6jd8v\" (UID: \"f721d6eb-f694-4e62-9133-12edd4f6c81c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6jd8v" Jul 22 01:37:40.439153 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.438858 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz842\" (UniqueName: \"kubernetes.io/projected/f721d6eb-f694-4e62-9133-12edd4f6c81c-kube-api-access-mz842\") pod \"kube-storage-version-migrator-operator-7658d9b749-6jd8v\" (UID: \"f721d6eb-f694-4e62-9133-12edd4f6c81c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6jd8v" Jul 22 01:37:40.439153 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.438951 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f7b6603-7aa5-411a-8956-40e0982d98f4-serving-cert\") pod \"insights-operator-555d9d7885-flhhx\" (UID: \"2f7b6603-7aa5-411a-8956-40e0982d98f4\") " pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.439153 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.438993 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f721d6eb-f694-4e62-9133-12edd4f6c81c-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-6jd8v\" (UID: \"f721d6eb-f694-4e62-9133-12edd4f6c81c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6jd8v" Jul 22 01:37:40.439153 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.439035 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzr9m\" (UniqueName: \"kubernetes.io/projected/01ab9b95-2508-4029-bd96-dcb5d29c5fb6-kube-api-access-pzr9m\") pod \"volume-data-source-validator-78cf4c6f6-68jls\" (UID: \"01ab9b95-2508-4029-bd96-dcb5d29c5fb6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-68jls" Jul 22 01:37:40.439153 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.439055 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f7b6603-7aa5-411a-8956-40e0982d98f4-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-flhhx\" (UID: \"2f7b6603-7aa5-411a-8956-40e0982d98f4\") " pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.441424 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.441406 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-fb4497b6-psq7s"] Jul 22 01:37:40.441541 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.441523 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" Jul 22 01:37:40.443860 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.443834 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 22 01:37:40.443989 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.443883 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 22 01:37:40.443989 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.443837 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 22 01:37:40.443989 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.443963 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 22 01:37:40.444244 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.444231 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-kxt2m\"" Jul 22 01:37:40.451091 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.451058 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 22 01:37:40.455652 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.455621 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-frfzq"] Jul 22 01:37:40.455652 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.455659 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wjp2"] Jul 22 01:37:40.455652 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.455679 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-z6wbm"] Jul 22 01:37:40.455652 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.455693 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-fb4497b6-psq7s"] Jul 22 01:37:40.455987 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.455705 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw"] Jul 22 01:37:40.455987 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.455800 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:40.458198 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.458171 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 22 01:37:40.458333 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.458266 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qs5rw\"" Jul 22 01:37:40.458582 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.458567 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 22 01:37:40.458664 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.458598 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 22 01:37:40.458664 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.458618 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 22 01:37:40.458664 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.458601 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 22 01:37:40.458816 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.458625 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 22 01:37:40.477336 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.477304 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8"] Jul 22 01:37:40.501678 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.501637 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8"] Jul 22 01:37:40.501837 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.501753 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8" Jul 22 01:37:40.504524 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.504501 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 22 01:37:40.504524 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.504532 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qn4sz\"" Jul 22 01:37:40.504715 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.504532 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 22 01:37:40.540153 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.540052 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjvcj\" (UniqueName: \"kubernetes.io/projected/6b7342d6-764b-45cf-b848-9e5a219b073c-kube-api-access-jjvcj\") pod \"service-ca-operator-5bcbc94c78-5wjp2\" (UID: \"6b7342d6-764b-45cf-b848-9e5a219b073c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wjp2" Jul 22 01:37:40.540153 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.540110 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f7b6603-7aa5-411a-8956-40e0982d98f4-serving-cert\") pod \"insights-operator-555d9d7885-flhhx\" (UID: \"2f7b6603-7aa5-411a-8956-40e0982d98f4\") " pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.540423 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.540170 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/546e33d1-6966-445c-9d8d-d10cf6631943-config\") pod \"console-operator-565b7b569d-z6wbm\" (UID: \"546e33d1-6966-445c-9d8d-d10cf6631943\") " pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" Jul 22 01:37:40.540423 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.540243 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzr9m\" (UniqueName: \"kubernetes.io/projected/01ab9b95-2508-4029-bd96-dcb5d29c5fb6-kube-api-access-pzr9m\") pod \"volume-data-source-validator-78cf4c6f6-68jls\" (UID: \"01ab9b95-2508-4029-bd96-dcb5d29c5fb6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-68jls" Jul 22 01:37:40.540423 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.540292 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f7b6603-7aa5-411a-8956-40e0982d98f4-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-flhhx\" (UID: \"2f7b6603-7aa5-411a-8956-40e0982d98f4\") " pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.540423 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.540318 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/53cff89d-d019-4bd3-b3ea-1eea796beb99-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-s4qrb\" (UID: \"53cff89d-d019-4bd3-b3ea-1eea796beb99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb" Jul 22 01:37:40.540423 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.540366 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75f76\" (UniqueName: \"kubernetes.io/projected/53cff89d-d019-4bd3-b3ea-1eea796beb99-kube-api-access-75f76\") pod \"cluster-samples-operator-79b7c55f87-s4qrb\" (UID: \"53cff89d-d019-4bd3-b3ea-1eea796beb99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb" Jul 22 01:37:40.540423 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.540399 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f7b6603-7aa5-411a-8956-40e0982d98f4-tmp\") pod \"insights-operator-555d9d7885-flhhx\" (UID: \"2f7b6603-7aa5-411a-8956-40e0982d98f4\") " pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.540700 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.540452 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2f7b6603-7aa5-411a-8956-40e0982d98f4-snapshots\") pod \"insights-operator-555d9d7885-flhhx\" (UID: \"2f7b6603-7aa5-411a-8956-40e0982d98f4\") " pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.540700 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.540487 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bhbr\" (UniqueName: \"kubernetes.io/projected/2f7b6603-7aa5-411a-8956-40e0982d98f4-kube-api-access-5bhbr\") pod \"insights-operator-555d9d7885-flhhx\" (UID: \"2f7b6603-7aa5-411a-8956-40e0982d98f4\") " pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.540700 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.540525 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl4dv\" (UniqueName: \"kubernetes.io/projected/a50a389d-59bd-4558-95b5-b5a3468291fc-kube-api-access-rl4dv\") pod \"network-check-source-7bc5b6bf85-frfzq\" (UID: \"a50a389d-59bd-4558-95b5-b5a3468291fc\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-frfzq" Jul 22 01:37:40.540700 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.540558 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6b7342d6-764b-45cf-b848-9e5a219b073c-config\") pod \"service-ca-operator-5bcbc94c78-5wjp2\" (UID: \"6b7342d6-764b-45cf-b848-9e5a219b073c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wjp2" Jul 22 01:37:40.540700 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.540591 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f721d6eb-f694-4e62-9133-12edd4f6c81c-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-6jd8v\" (UID: \"f721d6eb-f694-4e62-9133-12edd4f6c81c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6jd8v" Jul 22 01:37:40.540920 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.540760 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/546e33d1-6966-445c-9d8d-d10cf6631943-trusted-ca\") pod \"console-operator-565b7b569d-z6wbm\" (UID: \"546e33d1-6966-445c-9d8d-d10cf6631943\") " pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" Jul 22 01:37:40.540920 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.540815 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f721d6eb-f694-4e62-9133-12edd4f6c81c-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-6jd8v\" (UID: \"f721d6eb-f694-4e62-9133-12edd4f6c81c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6jd8v" Jul 22 01:37:40.540920 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.540842 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f7b6603-7aa5-411a-8956-40e0982d98f4-tmp\") pod \"insights-operator-555d9d7885-flhhx\" (UID: \"2f7b6603-7aa5-411a-8956-40e0982d98f4\") " pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.540920 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.540851 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz842\" (UniqueName: \"kubernetes.io/projected/f721d6eb-f694-4e62-9133-12edd4f6c81c-kube-api-access-mz842\") pod \"kube-storage-version-migrator-operator-7658d9b749-6jd8v\" (UID: \"f721d6eb-f694-4e62-9133-12edd4f6c81c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6jd8v" Jul 22 01:37:40.540920 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.540897 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f7b6603-7aa5-411a-8956-40e0982d98f4-service-ca-bundle\") pod \"insights-operator-555d9d7885-flhhx\" (UID: \"2f7b6603-7aa5-411a-8956-40e0982d98f4\") " pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.541114 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.540949 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6b7342d6-764b-45cf-b848-9e5a219b073c-serving-cert\") pod \"service-ca-operator-5bcbc94c78-5wjp2\" (UID: \"6b7342d6-764b-45cf-b848-9e5a219b073c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wjp2" Jul 22 01:37:40.541159 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.541108 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f721d6eb-f694-4e62-9133-12edd4f6c81c-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-6jd8v\" (UID: \"f721d6eb-f694-4e62-9133-12edd4f6c81c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6jd8v" Jul 22 01:37:40.541159 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.541146 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rswsx\" (UniqueName: \"kubernetes.io/projected/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-kube-api-access-rswsx\") pod \"cluster-monitoring-operator-76cf8966f8-6f9lw\" (UID: \"26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" Jul 22 01:37:40.541242 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.541180 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8hvj\" (UniqueName: \"kubernetes.io/projected/546e33d1-6966-445c-9d8d-d10cf6631943-kube-api-access-m8hvj\") pod \"console-operator-565b7b569d-z6wbm\" (UID: \"546e33d1-6966-445c-9d8d-d10cf6631943\") " pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" Jul 22 01:37:40.541242 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.541206 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-6f9lw\" (UID: \"26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" Jul 22 01:37:40.541242 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.541216 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2f7b6603-7aa5-411a-8956-40e0982d98f4-snapshots\") pod \"insights-operator-555d9d7885-flhhx\" (UID: \"2f7b6603-7aa5-411a-8956-40e0982d98f4\") " pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.541384 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.541246 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/546e33d1-6966-445c-9d8d-d10cf6631943-serving-cert\") pod \"console-operator-565b7b569d-z6wbm\" (UID: \"546e33d1-6966-445c-9d8d-d10cf6631943\") " pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" Jul 22 01:37:40.541384 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.541277 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-6f9lw\" (UID: \"26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" Jul 22 01:37:40.541384 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.541330 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f7b6603-7aa5-411a-8956-40e0982d98f4-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-flhhx\" (UID: \"2f7b6603-7aa5-411a-8956-40e0982d98f4\") " pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.541514 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.541432 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f7b6603-7aa5-411a-8956-40e0982d98f4-service-ca-bundle\") pod \"insights-operator-555d9d7885-flhhx\" (UID: \"2f7b6603-7aa5-411a-8956-40e0982d98f4\") " pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.544419 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.544385 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f721d6eb-f694-4e62-9133-12edd4f6c81c-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-6jd8v\" (UID: \"f721d6eb-f694-4e62-9133-12edd4f6c81c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6jd8v" Jul 22 01:37:40.544566 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.544491 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f7b6603-7aa5-411a-8956-40e0982d98f4-serving-cert\") pod \"insights-operator-555d9d7885-flhhx\" (UID: \"2f7b6603-7aa5-411a-8956-40e0982d98f4\") " pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.550459 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.550429 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz842\" (UniqueName: \"kubernetes.io/projected/f721d6eb-f694-4e62-9133-12edd4f6c81c-kube-api-access-mz842\") pod \"kube-storage-version-migrator-operator-7658d9b749-6jd8v\" (UID: \"f721d6eb-f694-4e62-9133-12edd4f6c81c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6jd8v" Jul 22 01:37:40.550459 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.550451 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bhbr\" (UniqueName: \"kubernetes.io/projected/2f7b6603-7aa5-411a-8956-40e0982d98f4-kube-api-access-5bhbr\") pod \"insights-operator-555d9d7885-flhhx\" (UID: \"2f7b6603-7aa5-411a-8956-40e0982d98f4\") " pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.550689 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.550461 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzr9m\" (UniqueName: \"kubernetes.io/projected/01ab9b95-2508-4029-bd96-dcb5d29c5fb6-kube-api-access-pzr9m\") pod \"volume-data-source-validator-78cf4c6f6-68jls\" (UID: \"01ab9b95-2508-4029-bd96-dcb5d29c5fb6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-68jls" Jul 22 01:37:40.626315 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.626267 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-68jls" Jul 22 01:37:40.641742 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.641708 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-flhhx" Jul 22 01:37:40.641885 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.641864 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/53cff89d-d019-4bd3-b3ea-1eea796beb99-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-s4qrb\" (UID: \"53cff89d-d019-4bd3-b3ea-1eea796beb99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb" Jul 22 01:37:40.641928 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.641896 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75f76\" (UniqueName: \"kubernetes.io/projected/53cff89d-d019-4bd3-b3ea-1eea796beb99-kube-api-access-75f76\") pod \"cluster-samples-operator-79b7c55f87-s4qrb\" (UID: \"53cff89d-d019-4bd3-b3ea-1eea796beb99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb" Jul 22 01:37:40.641928 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.641921 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-default-certificate\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:40.642024 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.641939 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:40.642024 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.641974 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl4dv\" (UniqueName: \"kubernetes.io/projected/a50a389d-59bd-4558-95b5-b5a3468291fc-kube-api-access-rl4dv\") pod \"network-check-source-7bc5b6bf85-frfzq\" (UID: \"a50a389d-59bd-4558-95b5-b5a3468291fc\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-frfzq" Jul 22 01:37:40.642024 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:40.641995 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 01:37:40.642152 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:40.642046 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53cff89d-d019-4bd3-b3ea-1eea796beb99-samples-operator-tls podName:53cff89d-d019-4bd3-b3ea-1eea796beb99 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:41.142030431 +0000 UTC m=+45.971278757 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/53cff89d-d019-4bd3-b3ea-1eea796beb99-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-s4qrb" (UID: "53cff89d-d019-4bd3-b3ea-1eea796beb99") : secret "samples-operator-tls" not found Jul 22 01:37:40.642152 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.642094 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6b7342d6-764b-45cf-b848-9e5a219b073c-config\") pod \"service-ca-operator-5bcbc94c78-5wjp2\" (UID: \"6b7342d6-764b-45cf-b848-9e5a219b073c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wjp2" Jul 22 01:37:40.642152 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.642134 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-bmvq8\" (UID: \"cc79c9c1-844a-4780-9aa1-b433eeea8848\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8" Jul 22 01:37:40.642559 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.642173 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/546e33d1-6966-445c-9d8d-d10cf6631943-trusted-ca\") pod \"console-operator-565b7b569d-z6wbm\" (UID: \"546e33d1-6966-445c-9d8d-d10cf6631943\") " pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" Jul 22 01:37:40.642559 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.642211 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48rs5\" (UniqueName: \"kubernetes.io/projected/e694641c-026f-4a5f-918b-6fedb0b96cdd-kube-api-access-48rs5\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:40.642559 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.642258 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:40.642559 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.642300 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6b7342d6-764b-45cf-b848-9e5a219b073c-serving-cert\") pod \"service-ca-operator-5bcbc94c78-5wjp2\" (UID: \"6b7342d6-764b-45cf-b848-9e5a219b073c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wjp2" Jul 22 01:37:40.642559 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.642365 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rswsx\" (UniqueName: \"kubernetes.io/projected/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-kube-api-access-rswsx\") pod \"cluster-monitoring-operator-76cf8966f8-6f9lw\" (UID: \"26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" Jul 22 01:37:40.642559 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.642392 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8hvj\" (UniqueName: \"kubernetes.io/projected/546e33d1-6966-445c-9d8d-d10cf6631943-kube-api-access-m8hvj\") pod \"console-operator-565b7b569d-z6wbm\" (UID: \"546e33d1-6966-445c-9d8d-d10cf6631943\") " pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" Jul 22 01:37:40.642559 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.642419 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-6f9lw\" (UID: \"26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" Jul 22 01:37:40.642559 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.642445 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/546e33d1-6966-445c-9d8d-d10cf6631943-serving-cert\") pod \"console-operator-565b7b569d-z6wbm\" (UID: \"546e33d1-6966-445c-9d8d-d10cf6631943\") " pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" Jul 22 01:37:40.642559 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.642472 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-6f9lw\" (UID: \"26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" Jul 22 01:37:40.642559 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.642505 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjvcj\" (UniqueName: \"kubernetes.io/projected/6b7342d6-764b-45cf-b848-9e5a219b073c-kube-api-access-jjvcj\") pod \"service-ca-operator-5bcbc94c78-5wjp2\" (UID: \"6b7342d6-764b-45cf-b848-9e5a219b073c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wjp2" Jul 22 01:37:40.643051 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:40.642583 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 01:37:40.643051 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:40.642654 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls podName:26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:41.142636371 +0000 UTC m=+45.971884710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-6f9lw" (UID: "26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1") : secret "cluster-monitoring-operator-tls" not found Jul 22 01:37:40.643051 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.642694 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-stats-auth\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:40.643051 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.642735 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/546e33d1-6966-445c-9d8d-d10cf6631943-config\") pod \"console-operator-565b7b569d-z6wbm\" (UID: \"546e33d1-6966-445c-9d8d-d10cf6631943\") " pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" Jul 22 01:37:40.643051 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.642744 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6b7342d6-764b-45cf-b848-9e5a219b073c-config\") pod \"service-ca-operator-5bcbc94c78-5wjp2\" (UID: \"6b7342d6-764b-45cf-b848-9e5a219b073c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wjp2" Jul 22 01:37:40.643051 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.642765 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cc79c9c1-844a-4780-9aa1-b433eeea8848-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-bmvq8\" (UID: \"cc79c9c1-844a-4780-9aa1-b433eeea8848\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8" Jul 22 01:37:40.643281 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.643140 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/546e33d1-6966-445c-9d8d-d10cf6631943-trusted-ca\") pod \"console-operator-565b7b569d-z6wbm\" (UID: \"546e33d1-6966-445c-9d8d-d10cf6631943\") " pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" Jul 22 01:37:40.643383 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.643363 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/546e33d1-6966-445c-9d8d-d10cf6631943-config\") pod \"console-operator-565b7b569d-z6wbm\" (UID: \"546e33d1-6966-445c-9d8d-d10cf6631943\") " pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" Jul 22 01:37:40.643433 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.643397 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-6f9lw\" (UID: \"26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" Jul 22 01:37:40.645173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.645146 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/546e33d1-6966-445c-9d8d-d10cf6631943-serving-cert\") pod \"console-operator-565b7b569d-z6wbm\" (UID: \"546e33d1-6966-445c-9d8d-d10cf6631943\") " pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" Jul 22 01:37:40.645268 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.645145 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6b7342d6-764b-45cf-b848-9e5a219b073c-serving-cert\") pod \"service-ca-operator-5bcbc94c78-5wjp2\" (UID: \"6b7342d6-764b-45cf-b848-9e5a219b073c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wjp2" Jul 22 01:37:40.654729 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.654699 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjvcj\" (UniqueName: \"kubernetes.io/projected/6b7342d6-764b-45cf-b848-9e5a219b073c-kube-api-access-jjvcj\") pod \"service-ca-operator-5bcbc94c78-5wjp2\" (UID: \"6b7342d6-764b-45cf-b848-9e5a219b073c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wjp2" Jul 22 01:37:40.654855 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.654809 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8hvj\" (UniqueName: \"kubernetes.io/projected/546e33d1-6966-445c-9d8d-d10cf6631943-kube-api-access-m8hvj\") pod \"console-operator-565b7b569d-z6wbm\" (UID: \"546e33d1-6966-445c-9d8d-d10cf6631943\") " pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" Jul 22 01:37:40.654926 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.654875 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6jd8v" Jul 22 01:37:40.655286 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.655266 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75f76\" (UniqueName: \"kubernetes.io/projected/53cff89d-d019-4bd3-b3ea-1eea796beb99-kube-api-access-75f76\") pod \"cluster-samples-operator-79b7c55f87-s4qrb\" (UID: \"53cff89d-d019-4bd3-b3ea-1eea796beb99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb" Jul 22 01:37:40.655449 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.655426 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rswsx\" (UniqueName: \"kubernetes.io/projected/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-kube-api-access-rswsx\") pod \"cluster-monitoring-operator-76cf8966f8-6f9lw\" (UID: \"26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" Jul 22 01:37:40.655566 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.655550 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl4dv\" (UniqueName: \"kubernetes.io/projected/a50a389d-59bd-4558-95b5-b5a3468291fc-kube-api-access-rl4dv\") pod \"network-check-source-7bc5b6bf85-frfzq\" (UID: \"a50a389d-59bd-4558-95b5-b5a3468291fc\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-frfzq" Jul 22 01:37:40.698753 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.698361 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-frfzq" Jul 22 01:37:40.728290 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.728247 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wjp2" Jul 22 01:37:40.743576 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.743416 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48rs5\" (UniqueName: \"kubernetes.io/projected/e694641c-026f-4a5f-918b-6fedb0b96cdd-kube-api-access-48rs5\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:40.743576 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.743465 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:40.743576 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.743554 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-stats-auth\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:40.743576 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.743582 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cc79c9c1-844a-4780-9aa1-b433eeea8848-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-bmvq8\" (UID: \"cc79c9c1-844a-4780-9aa1-b433eeea8848\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8" Jul 22 01:37:40.743918 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.743656 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-default-certificate\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:40.743918 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.743679 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:40.743918 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.743740 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-bmvq8\" (UID: \"cc79c9c1-844a-4780-9aa1-b433eeea8848\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8" Jul 22 01:37:40.743918 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:40.743892 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 01:37:40.744110 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:40.743956 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert podName:cc79c9c1-844a-4780-9aa1-b433eeea8848 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:41.243936269 +0000 UTC m=+46.073184601 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-bmvq8" (UID: "cc79c9c1-844a-4780-9aa1-b433eeea8848") : secret "networking-console-plugin-cert" not found Jul 22 01:37:40.745008 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:40.744519 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle podName:e694641c-026f-4a5f-918b-6fedb0b96cdd nodeName:}" failed. No retries permitted until 2026-07-22 01:37:41.244500367 +0000 UTC m=+46.073748710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle") pod "router-default-fb4497b6-psq7s" (UID: "e694641c-026f-4a5f-918b-6fedb0b96cdd") : configmap references non-existent config key: service-ca.crt Jul 22 01:37:40.745286 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:40.745250 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 01:37:40.745360 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:40.745319 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs podName:e694641c-026f-4a5f-918b-6fedb0b96cdd nodeName:}" failed. No retries permitted until 2026-07-22 01:37:41.245301026 +0000 UTC m=+46.074549352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs") pod "router-default-fb4497b6-psq7s" (UID: "e694641c-026f-4a5f-918b-6fedb0b96cdd") : secret "router-metrics-certs-default" not found Jul 22 01:37:40.746124 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.746069 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cc79c9c1-844a-4780-9aa1-b433eeea8848-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-bmvq8\" (UID: \"cc79c9c1-844a-4780-9aa1-b433eeea8848\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8" Jul 22 01:37:40.747775 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.747747 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-stats-auth\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:40.748722 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.748699 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-default-certificate\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:40.753747 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.753553 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" Jul 22 01:37:40.755809 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.755783 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48rs5\" (UniqueName: \"kubernetes.io/projected/e694641c-026f-4a5f-918b-6fedb0b96cdd-kube-api-access-48rs5\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:40.869572 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.866913 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-68jls"] Jul 22 01:37:40.883508 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.882932 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-flhhx"] Jul 22 01:37:40.888460 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:37:40.888426 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f7b6603_7aa5_411a_8956_40e0982d98f4.slice/crio-bf26d8b916aa1f9ea22e72c1a8999ae1ab8c81d2ffdc7dc9f0be8617aac4cfbc WatchSource:0}: Error finding container bf26d8b916aa1f9ea22e72c1a8999ae1ab8c81d2ffdc7dc9f0be8617aac4cfbc: Status 404 returned error can't find the container with id bf26d8b916aa1f9ea22e72c1a8999ae1ab8c81d2ffdc7dc9f0be8617aac4cfbc Jul 22 01:37:40.912846 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.912759 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6jd8v"] Jul 22 01:37:40.917487 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:37:40.917450 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf721d6eb_f694_4e62_9133_12edd4f6c81c.slice/crio-15d3aa3e88a16b4bbb3375f4525870107536371b463ca4a56f77eba3640c0a4e WatchSource:0}: Error finding container 15d3aa3e88a16b4bbb3375f4525870107536371b463ca4a56f77eba3640c0a4e: Status 404 returned error can't find the container with id 15d3aa3e88a16b4bbb3375f4525870107536371b463ca4a56f77eba3640c0a4e Jul 22 01:37:40.941387 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.941339 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-frfzq"] Jul 22 01:37:40.944212 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:37:40.944177 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda50a389d_59bd_4558_95b5_b5a3468291fc.slice/crio-b5bac077d20bb8836517d664f22e27a2220f0fa0b3d0dced2802a766b3ffc304 WatchSource:0}: Error finding container b5bac077d20bb8836517d664f22e27a2220f0fa0b3d0dced2802a766b3ffc304: Status 404 returned error can't find the container with id b5bac077d20bb8836517d664f22e27a2220f0fa0b3d0dced2802a766b3ffc304 Jul 22 01:37:40.949299 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.949272 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wjp2"] Jul 22 01:37:40.952692 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:37:40.952665 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b7342d6_764b_45cf_b848_9e5a219b073c.slice/crio-172a33a2445590f6f34c0928288981ebdf767a719c99ce10f722414e31bf62af WatchSource:0}: Error finding container 172a33a2445590f6f34c0928288981ebdf767a719c99ce10f722414e31bf62af: Status 404 returned error can't find the container with id 172a33a2445590f6f34c0928288981ebdf767a719c99ce10f722414e31bf62af Jul 22 01:37:40.961086 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.961052 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-frfzq" event={"ID":"a50a389d-59bd-4558-95b5-b5a3468291fc","Type":"ContainerStarted","Data":"b5bac077d20bb8836517d664f22e27a2220f0fa0b3d0dced2802a766b3ffc304"} Jul 22 01:37:40.973979 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.973907 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-z6wbm"] Jul 22 01:37:40.975046 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.975014 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6jd8v" event={"ID":"f721d6eb-f694-4e62-9133-12edd4f6c81c","Type":"ContainerStarted","Data":"15d3aa3e88a16b4bbb3375f4525870107536371b463ca4a56f77eba3640c0a4e"} Jul 22 01:37:40.976447 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.976424 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-68jls" event={"ID":"01ab9b95-2508-4029-bd96-dcb5d29c5fb6","Type":"ContainerStarted","Data":"aaaf14dc2b3ab9eec1a2ba7348022b09a5816aff906af92ab4d8221c9729e647"} Jul 22 01:37:40.977601 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.977574 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wjp2" event={"ID":"6b7342d6-764b-45cf-b848-9e5a219b073c","Type":"ContainerStarted","Data":"172a33a2445590f6f34c0928288981ebdf767a719c99ce10f722414e31bf62af"} Jul 22 01:37:40.978434 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:37:40.978402 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod546e33d1_6966_445c_9d8d_d10cf6631943.slice/crio-0c67ffd0645d37f2c4b2d25c2323b782cd613f9647a3e4e32d3748fa72387b67 WatchSource:0}: Error finding container 0c67ffd0645d37f2c4b2d25c2323b782cd613f9647a3e4e32d3748fa72387b67: Status 404 returned error can't find the container with id 0c67ffd0645d37f2c4b2d25c2323b782cd613f9647a3e4e32d3748fa72387b67 Jul 22 01:37:40.978634 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:40.978610 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-flhhx" event={"ID":"2f7b6603-7aa5-411a-8956-40e0982d98f4","Type":"ContainerStarted","Data":"bf26d8b916aa1f9ea22e72c1a8999ae1ab8c81d2ffdc7dc9f0be8617aac4cfbc"} Jul 22 01:37:41.148059 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:41.148013 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-6f9lw\" (UID: \"26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" Jul 22 01:37:41.148272 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:41.148107 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/53cff89d-d019-4bd3-b3ea-1eea796beb99-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-s4qrb\" (UID: \"53cff89d-d019-4bd3-b3ea-1eea796beb99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb" Jul 22 01:37:41.148272 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:41.148177 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 01:37:41.148272 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:41.148241 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 01:37:41.148422 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:41.148251 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls podName:26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:42.148234175 +0000 UTC m=+46.977482501 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-6f9lw" (UID: "26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1") : secret "cluster-monitoring-operator-tls" not found Jul 22 01:37:41.148422 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:41.148299 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53cff89d-d019-4bd3-b3ea-1eea796beb99-samples-operator-tls podName:53cff89d-d019-4bd3-b3ea-1eea796beb99 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:42.148286293 +0000 UTC m=+46.977534619 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/53cff89d-d019-4bd3-b3ea-1eea796beb99-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-s4qrb" (UID: "53cff89d-d019-4bd3-b3ea-1eea796beb99") : secret "samples-operator-tls" not found Jul 22 01:37:41.249479 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:41.249445 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:41.249633 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:41.249620 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:41.249702 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:41.249661 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-bmvq8\" (UID: \"cc79c9c1-844a-4780-9aa1-b433eeea8848\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8" Jul 22 01:37:41.249754 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:41.249735 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle podName:e694641c-026f-4a5f-918b-6fedb0b96cdd nodeName:}" failed. No retries permitted until 2026-07-22 01:37:42.249694483 +0000 UTC m=+47.078942824 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle") pod "router-default-fb4497b6-psq7s" (UID: "e694641c-026f-4a5f-918b-6fedb0b96cdd") : configmap references non-existent config key: service-ca.crt Jul 22 01:37:41.249804 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:41.249762 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 01:37:41.249804 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:41.249763 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 01:37:41.249804 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:41.249803 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert podName:cc79c9c1-844a-4780-9aa1-b433eeea8848 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:42.249791313 +0000 UTC m=+47.079039638 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-bmvq8" (UID: "cc79c9c1-844a-4780-9aa1-b433eeea8848") : secret "networking-console-plugin-cert" not found Jul 22 01:37:41.249887 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:41.249815 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs podName:e694641c-026f-4a5f-918b-6fedb0b96cdd nodeName:}" failed. No retries permitted until 2026-07-22 01:37:42.249809318 +0000 UTC m=+47.079057643 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs") pod "router-default-fb4497b6-psq7s" (UID: "e694641c-026f-4a5f-918b-6fedb0b96cdd") : secret "router-metrics-certs-default" not found Jul 22 01:37:41.985890 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:41.985774 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" event={"ID":"546e33d1-6966-445c-9d8d-d10cf6631943","Type":"ContainerStarted","Data":"0c67ffd0645d37f2c4b2d25c2323b782cd613f9647a3e4e32d3748fa72387b67"} Jul 22 01:37:42.161013 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:42.160099 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-6f9lw\" (UID: \"26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" Jul 22 01:37:42.161013 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:42.160201 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/53cff89d-d019-4bd3-b3ea-1eea796beb99-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-s4qrb\" (UID: \"53cff89d-d019-4bd3-b3ea-1eea796beb99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb" Jul 22 01:37:42.161013 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:42.160436 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 01:37:42.161013 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:42.160501 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53cff89d-d019-4bd3-b3ea-1eea796beb99-samples-operator-tls podName:53cff89d-d019-4bd3-b3ea-1eea796beb99 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:44.160481629 +0000 UTC m=+48.989729970 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/53cff89d-d019-4bd3-b3ea-1eea796beb99-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-s4qrb" (UID: "53cff89d-d019-4bd3-b3ea-1eea796beb99") : secret "samples-operator-tls" not found Jul 22 01:37:42.161013 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:42.160897 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 01:37:42.161013 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:42.160952 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls podName:26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:44.160933437 +0000 UTC m=+48.990181779 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-6f9lw" (UID: "26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1") : secret "cluster-monitoring-operator-tls" not found Jul 22 01:37:42.261052 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:42.260959 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:42.261052 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:42.261043 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-bmvq8\" (UID: \"cc79c9c1-844a-4780-9aa1-b433eeea8848\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8" Jul 22 01:37:42.261293 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:42.261100 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:42.261293 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:42.261255 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle podName:e694641c-026f-4a5f-918b-6fedb0b96cdd nodeName:}" failed. No retries permitted until 2026-07-22 01:37:44.261235735 +0000 UTC m=+49.090484064 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle") pod "router-default-fb4497b6-psq7s" (UID: "e694641c-026f-4a5f-918b-6fedb0b96cdd") : configmap references non-existent config key: service-ca.crt Jul 22 01:37:42.261445 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:42.261388 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 01:37:42.261445 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:42.261436 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert podName:cc79c9c1-844a-4780-9aa1-b433eeea8848 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:44.261421584 +0000 UTC m=+49.090669915 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-bmvq8" (UID: "cc79c9c1-844a-4780-9aa1-b433eeea8848") : secret "networking-console-plugin-cert" not found Jul 22 01:37:42.261554 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:42.261492 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 01:37:42.261554 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:42.261523 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs podName:e694641c-026f-4a5f-918b-6fedb0b96cdd nodeName:}" failed. No retries permitted until 2026-07-22 01:37:44.261513096 +0000 UTC m=+49.090761424 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs") pod "router-default-fb4497b6-psq7s" (UID: "e694641c-026f-4a5f-918b-6fedb0b96cdd") : secret "router-metrics-certs-default" not found Jul 22 01:37:43.994170 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:43.994126 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-68jls" event={"ID":"01ab9b95-2508-4029-bd96-dcb5d29c5fb6","Type":"ContainerStarted","Data":"7d9f569c784e7135c54d7e41c8350290fb2d6c18a29fdf5c4cd151af595cfe53"} Jul 22 01:37:44.008879 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:44.008708 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-68jls" podStartSLOduration=1.924272159 podStartE2EDuration="4.008686254s" totalImagesPullingTime="2.084414095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:37:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:44.008124062 +0000 UTC m=+48.837372411" watchObservedRunningTime="2026-07-22 01:37:44.008686254 +0000 UTC m=+48.837934603" Jul 22 01:37:44.187312 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:44.187265 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/53cff89d-d019-4bd3-b3ea-1eea796beb99-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-s4qrb\" (UID: \"53cff89d-d019-4bd3-b3ea-1eea796beb99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb" Jul 22 01:37:44.187534 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:44.187459 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-6f9lw\" (UID: \"26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" Jul 22 01:37:44.187534 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:44.187467 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 01:37:44.187651 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:44.187549 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53cff89d-d019-4bd3-b3ea-1eea796beb99-samples-operator-tls podName:53cff89d-d019-4bd3-b3ea-1eea796beb99 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:48.187526503 +0000 UTC m=+53.016774846 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/53cff89d-d019-4bd3-b3ea-1eea796beb99-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-s4qrb" (UID: "53cff89d-d019-4bd3-b3ea-1eea796beb99") : secret "samples-operator-tls" not found Jul 22 01:37:44.187651 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:44.187573 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 01:37:44.187747 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:44.187652 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls podName:26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:48.1876334 +0000 UTC m=+53.016881740 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-6f9lw" (UID: "26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1") : secret "cluster-monitoring-operator-tls" not found Jul 22 01:37:44.288889 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:44.288752 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:44.288889 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:44.288817 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-bmvq8\" (UID: \"cc79c9c1-844a-4780-9aa1-b433eeea8848\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8" Jul 22 01:37:44.288889 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:44.288899 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 01:37:44.289160 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:44.288908 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 01:37:44.289160 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:44.288957 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert podName:cc79c9c1-844a-4780-9aa1-b433eeea8848 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:48.288943911 +0000 UTC m=+53.118192241 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-bmvq8" (UID: "cc79c9c1-844a-4780-9aa1-b433eeea8848") : secret "networking-console-plugin-cert" not found Jul 22 01:37:44.289160 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:44.288987 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs podName:e694641c-026f-4a5f-918b-6fedb0b96cdd nodeName:}" failed. No retries permitted until 2026-07-22 01:37:48.288969113 +0000 UTC m=+53.118217444 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs") pod "router-default-fb4497b6-psq7s" (UID: "e694641c-026f-4a5f-918b-6fedb0b96cdd") : secret "router-metrics-certs-default" not found Jul 22 01:37:44.289160 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:44.289034 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:44.289298 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:44.289172 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle podName:e694641c-026f-4a5f-918b-6fedb0b96cdd nodeName:}" failed. No retries permitted until 2026-07-22 01:37:48.2891564 +0000 UTC m=+53.118404746 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle") pod "router-default-fb4497b6-psq7s" (UID: "e694641c-026f-4a5f-918b-6fedb0b96cdd") : configmap references non-existent config key: service-ca.crt Jul 22 01:37:46.915702 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:46.915663 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:37:46.916086 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:46.915839 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 01:37:46.916086 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:46.915861 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-687855c694-q2wpf: secret "image-registry-tls" not found Jul 22 01:37:46.916086 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:46.915921 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls podName:8ea58868-1c24-48f4-8c3d-2427d0a6dbe2 nodeName:}" failed. No retries permitted until 2026-07-22 01:38:02.915905192 +0000 UTC m=+67.745153522 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls") pod "image-registry-687855c694-q2wpf" (UID: "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2") : secret "image-registry-tls" not found Jul 22 01:37:47.017039 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:47.017001 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls\") pod \"dns-default-g2zdj\" (UID: \"cb80e90e-d908-490b-a9da-bb1d624d6615\") " pod="openshift-dns/dns-default-g2zdj" Jul 22 01:37:47.017215 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:47.017195 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 01:37:47.017274 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:47.017265 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls podName:cb80e90e-d908-490b-a9da-bb1d624d6615 nodeName:}" failed. No retries permitted until 2026-07-22 01:38:03.017249146 +0000 UTC m=+67.846497475 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls") pod "dns-default-g2zdj" (UID: "cb80e90e-d908-490b-a9da-bb1d624d6615") : secret "dns-default-metrics-tls" not found Jul 22 01:37:48.007721 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.007510 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wjp2" event={"ID":"6b7342d6-764b-45cf-b848-9e5a219b073c","Type":"ContainerStarted","Data":"3b78f846c7ce212d8ce410fede754d785e172ad488930027e8c5095d8d6f5cf9"} Jul 22 01:37:48.009614 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.009584 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/0.log" Jul 22 01:37:48.009759 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.009634 2559 generic.go:356] "Generic (PLEG): container finished" podID="546e33d1-6966-445c-9d8d-d10cf6631943" containerID="63e94dcf30aeb3155c29fb3bbb7e525fb224975420fc03ee1f0beae83ee18580" exitCode=255 Jul 22 01:37:48.009833 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.009743 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" event={"ID":"546e33d1-6966-445c-9d8d-d10cf6631943","Type":"ContainerDied","Data":"63e94dcf30aeb3155c29fb3bbb7e525fb224975420fc03ee1f0beae83ee18580"} Jul 22 01:37:48.009987 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.009969 2559 scope.go:117] "RemoveContainer" containerID="63e94dcf30aeb3155c29fb3bbb7e525fb224975420fc03ee1f0beae83ee18580" Jul 22 01:37:48.011844 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.011815 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-flhhx" event={"ID":"2f7b6603-7aa5-411a-8956-40e0982d98f4","Type":"ContainerStarted","Data":"2751b2a5511bb1983de4623727c3845a9d278b2a7a68ee937cf9b88256a3d5e9"} Jul 22 01:37:48.014117 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.014065 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-frfzq" event={"ID":"a50a389d-59bd-4558-95b5-b5a3468291fc","Type":"ContainerStarted","Data":"134f32b466d39ccc37b7576c016377a5bef8dc6df03ba0c9bcd2a9844e5e2ed7"} Jul 22 01:37:48.015626 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.015584 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6jd8v" event={"ID":"f721d6eb-f694-4e62-9133-12edd4f6c81c","Type":"ContainerStarted","Data":"bd771db56ff5e30690d328dc384aec9f090ec745f349f1a63bf807ae8300660b"} Jul 22 01:37:48.024189 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.024121 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wjp2" podStartSLOduration=1.617249699 podStartE2EDuration="8.02410158s" totalImagesPullingTime="6.406851881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:37:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:48.02285716 +0000 UTC m=+52.852105505" watchObservedRunningTime="2026-07-22 01:37:48.02410158 +0000 UTC m=+52.853349929" Jul 22 01:37:48.040518 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.040107 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6jd8v" podStartSLOduration=1.598863491 podStartE2EDuration="8.040084429s" totalImagesPullingTime="6.441220938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:37:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:48.038320864 +0000 UTC m=+52.867569213" watchObservedRunningTime="2026-07-22 01:37:48.040084429 +0000 UTC m=+52.869332774" Jul 22 01:37:48.058785 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.058706 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-frfzq" podStartSLOduration=1.633415257 podStartE2EDuration="8.058685386s" totalImagesPullingTime="6.425270129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:37:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:48.057495415 +0000 UTC m=+52.886743762" watchObservedRunningTime="2026-07-22 01:37:48.058685386 +0000 UTC m=+52.887933735" Jul 22 01:37:48.231640 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.231581 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/53cff89d-d019-4bd3-b3ea-1eea796beb99-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-s4qrb\" (UID: \"53cff89d-d019-4bd3-b3ea-1eea796beb99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb" Jul 22 01:37:48.231800 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:48.231731 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 01:37:48.231800 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.231760 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-6f9lw\" (UID: \"26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" Jul 22 01:37:48.231918 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:48.231805 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53cff89d-d019-4bd3-b3ea-1eea796beb99-samples-operator-tls podName:53cff89d-d019-4bd3-b3ea-1eea796beb99 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:56.231781741 +0000 UTC m=+61.061030140 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/53cff89d-d019-4bd3-b3ea-1eea796beb99-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-s4qrb" (UID: "53cff89d-d019-4bd3-b3ea-1eea796beb99") : secret "samples-operator-tls" not found Jul 22 01:37:48.231981 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:48.231922 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 01:37:48.232034 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:48.232004 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls podName:26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:56.231983828 +0000 UTC m=+61.061232163 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-6f9lw" (UID: "26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1") : secret "cluster-monitoring-operator-tls" not found Jul 22 01:37:48.333170 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.333131 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:48.333394 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.333214 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-bmvq8\" (UID: \"cc79c9c1-844a-4780-9aa1-b433eeea8848\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8" Jul 22 01:37:48.333394 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.333265 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:48.333394 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:48.333313 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 01:37:48.333394 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:48.333333 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 01:37:48.333594 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:48.333404 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs podName:e694641c-026f-4a5f-918b-6fedb0b96cdd nodeName:}" failed. No retries permitted until 2026-07-22 01:37:56.333383589 +0000 UTC m=+61.162631921 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs") pod "router-default-fb4497b6-psq7s" (UID: "e694641c-026f-4a5f-918b-6fedb0b96cdd") : secret "router-metrics-certs-default" not found Jul 22 01:37:48.333594 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:48.333423 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert podName:cc79c9c1-844a-4780-9aa1-b433eeea8848 nodeName:}" failed. No retries permitted until 2026-07-22 01:37:56.333414217 +0000 UTC m=+61.162662549 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-bmvq8" (UID: "cc79c9c1-844a-4780-9aa1-b433eeea8848") : secret "networking-console-plugin-cert" not found Jul 22 01:37:48.333594 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:48.333439 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle podName:e694641c-026f-4a5f-918b-6fedb0b96cdd nodeName:}" failed. No retries permitted until 2026-07-22 01:37:56.3334297 +0000 UTC m=+61.162678028 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle") pod "router-default-fb4497b6-psq7s" (UID: "e694641c-026f-4a5f-918b-6fedb0b96cdd") : configmap references non-existent config key: service-ca.crt Jul 22 01:37:48.687218 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.687100 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-flhhx" podStartSLOduration=2.216303879 podStartE2EDuration="8.687076346s" totalImagesPullingTime="6.470772467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:37:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:48.089203931 +0000 UTC m=+52.918452275" watchObservedRunningTime="2026-07-22 01:37:48.687076346 +0000 UTC m=+53.516324695" Jul 22 01:37:48.689165 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.689130 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gdxvd"] Jul 22 01:37:48.702897 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.702868 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:48.705671 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.705638 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 01:37:48.707542 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.707505 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gdxvd"] Jul 22 01:37:48.707822 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.707800 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-l77sq\"" Jul 22 01:37:48.707921 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.707899 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 01:37:48.738037 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.737992 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d6d3be9e-b32f-4f80-b49f-38854284252f-data-volume\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:48.738208 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.738053 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d6d3be9e-b32f-4f80-b49f-38854284252f-crio-socket\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:48.738208 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.738090 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4g9z\" (UniqueName: \"kubernetes.io/projected/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-api-access-m4g9z\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:48.738208 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.738122 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:48.738208 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.738180 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:48.839138 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.839085 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d6d3be9e-b32f-4f80-b49f-38854284252f-data-volume\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:48.839316 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.839178 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d6d3be9e-b32f-4f80-b49f-38854284252f-crio-socket\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:48.839316 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.839220 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4g9z\" (UniqueName: \"kubernetes.io/projected/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-api-access-m4g9z\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:48.839316 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.839278 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:48.839480 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.839332 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:48.839480 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.839434 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d6d3be9e-b32f-4f80-b49f-38854284252f-crio-socket\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:48.839480 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:48.839458 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 01:37:48.839480 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:48.839477 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 01:37:48.839601 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:48.839529 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-rbac-proxy-cm podName:d6d3be9e-b32f-4f80-b49f-38854284252f nodeName:}" failed. No retries permitted until 2026-07-22 01:37:49.339510071 +0000 UTC m=+54.168758399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gdxvd" (UID: "d6d3be9e-b32f-4f80-b49f-38854284252f") : configmap "kube-rbac-proxy" not found Jul 22 01:37:48.839601 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.839525 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d6d3be9e-b32f-4f80-b49f-38854284252f-data-volume\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:48.839601 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:48.839543 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls podName:d6d3be9e-b32f-4f80-b49f-38854284252f nodeName:}" failed. No retries permitted until 2026-07-22 01:37:49.339537021 +0000 UTC m=+54.168785346 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gdxvd" (UID: "d6d3be9e-b32f-4f80-b49f-38854284252f") : secret "insights-runtime-extractor-tls" not found Jul 22 01:37:48.849044 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:48.849008 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4g9z\" (UniqueName: \"kubernetes.io/projected/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-api-access-m4g9z\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:49.021621 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:49.021545 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 01:37:49.022063 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:49.021999 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/0.log" Jul 22 01:37:49.022063 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:49.022037 2559 generic.go:356] "Generic (PLEG): container finished" podID="546e33d1-6966-445c-9d8d-d10cf6631943" containerID="39c66b4c0fd7dbc19cb4ef10b37261e3a924a60a43885b2df72b0fe461ff2cc9" exitCode=255 Jul 22 01:37:49.022152 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:49.022125 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" event={"ID":"546e33d1-6966-445c-9d8d-d10cf6631943","Type":"ContainerDied","Data":"39c66b4c0fd7dbc19cb4ef10b37261e3a924a60a43885b2df72b0fe461ff2cc9"} Jul 22 01:37:49.022247 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:49.022224 2559 scope.go:117] "RemoveContainer" containerID="63e94dcf30aeb3155c29fb3bbb7e525fb224975420fc03ee1f0beae83ee18580" Jul 22 01:37:49.022575 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:49.022447 2559 scope.go:117] "RemoveContainer" containerID="39c66b4c0fd7dbc19cb4ef10b37261e3a924a60a43885b2df72b0fe461ff2cc9" Jul 22 01:37:49.022682 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:49.022625 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-z6wbm_openshift-console-operator(546e33d1-6966-445c-9d8d-d10cf6631943)\"" pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" podUID="546e33d1-6966-445c-9d8d-d10cf6631943" Jul 22 01:37:49.344323 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:49.344281 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:49.344577 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:49.344362 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:49.344577 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:49.344441 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 01:37:49.344577 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:49.344485 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 01:37:49.344577 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:49.344520 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-rbac-proxy-cm podName:d6d3be9e-b32f-4f80-b49f-38854284252f nodeName:}" failed. No retries permitted until 2026-07-22 01:37:50.34450111 +0000 UTC m=+55.173749449 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gdxvd" (UID: "d6d3be9e-b32f-4f80-b49f-38854284252f") : configmap "kube-rbac-proxy" not found Jul 22 01:37:49.344577 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:49.344536 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls podName:d6d3be9e-b32f-4f80-b49f-38854284252f nodeName:}" failed. No retries permitted until 2026-07-22 01:37:50.344530003 +0000 UTC m=+55.173778332 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gdxvd" (UID: "d6d3be9e-b32f-4f80-b49f-38854284252f") : secret "insights-runtime-extractor-tls" not found Jul 22 01:37:50.026451 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:50.026419 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 01:37:50.026850 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:50.026767 2559 scope.go:117] "RemoveContainer" containerID="39c66b4c0fd7dbc19cb4ef10b37261e3a924a60a43885b2df72b0fe461ff2cc9" Jul 22 01:37:50.026946 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:50.026925 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-z6wbm_openshift-console-operator(546e33d1-6966-445c-9d8d-d10cf6631943)\"" pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" podUID="546e33d1-6966-445c-9d8d-d10cf6631943" Jul 22 01:37:50.353155 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:50.353115 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:50.353330 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:50.353179 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:50.353330 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:50.353239 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 01:37:50.353330 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:50.353320 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-rbac-proxy-cm podName:d6d3be9e-b32f-4f80-b49f-38854284252f nodeName:}" failed. No retries permitted until 2026-07-22 01:37:52.353300424 +0000 UTC m=+57.182548750 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gdxvd" (UID: "d6d3be9e-b32f-4f80-b49f-38854284252f") : configmap "kube-rbac-proxy" not found Jul 22 01:37:50.353497 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:50.353335 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 01:37:50.353497 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:50.353396 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls podName:d6d3be9e-b32f-4f80-b49f-38854284252f nodeName:}" failed. No retries permitted until 2026-07-22 01:37:52.35338477 +0000 UTC m=+57.182633115 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gdxvd" (UID: "d6d3be9e-b32f-4f80-b49f-38854284252f") : secret "insights-runtime-extractor-tls" not found Jul 22 01:37:50.754153 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:50.754062 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" Jul 22 01:37:50.754153 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:50.754129 2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" Jul 22 01:37:50.990336 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:50.990299 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zvt6m_acaa1746-a8a5-4517-9299-9a502758b7a4/dns-node-resolver/0.log" Jul 22 01:37:51.029804 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:51.029729 2559 scope.go:117] "RemoveContainer" containerID="39c66b4c0fd7dbc19cb4ef10b37261e3a924a60a43885b2df72b0fe461ff2cc9" Jul 22 01:37:51.030155 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:51.029891 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-z6wbm_openshift-console-operator(546e33d1-6966-445c-9d8d-d10cf6631943)\"" pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" podUID="546e33d1-6966-445c-9d8d-d10cf6631943" Jul 22 01:37:51.991447 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:51.991414 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rs2vg_e3459ca6-711d-4b8a-b1d1-3f65d03ce297/node-ca/0.log" Jul 22 01:37:52.374563 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:52.374522 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:52.375070 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:52.374587 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:52.375070 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:52.374669 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 01:37:52.375070 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:52.374749 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 01:37:52.375070 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:52.374765 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-rbac-proxy-cm podName:d6d3be9e-b32f-4f80-b49f-38854284252f nodeName:}" failed. No retries permitted until 2026-07-22 01:37:56.374742942 +0000 UTC m=+61.203991288 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gdxvd" (UID: "d6d3be9e-b32f-4f80-b49f-38854284252f") : configmap "kube-rbac-proxy" not found Jul 22 01:37:52.375070 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:52.374807 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls podName:d6d3be9e-b32f-4f80-b49f-38854284252f nodeName:}" failed. No retries permitted until 2026-07-22 01:37:56.374794241 +0000 UTC m=+61.204042566 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gdxvd" (UID: "d6d3be9e-b32f-4f80-b49f-38854284252f") : secret "insights-runtime-extractor-tls" not found Jul 22 01:37:52.792362 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:52.792222 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qhr55_0df0a233-5828-4368-96ca-134ab7641aed/serve-healthcheck-canary/0.log" Jul 22 01:37:56.307566 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:56.307523 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-6f9lw\" (UID: \"26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" Jul 22 01:37:56.308245 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:56.307677 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/53cff89d-d019-4bd3-b3ea-1eea796beb99-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-s4qrb\" (UID: \"53cff89d-d019-4bd3-b3ea-1eea796beb99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb" Jul 22 01:37:56.308245 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:56.307682 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 01:37:56.308245 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:56.307761 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls podName:26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1 nodeName:}" failed. No retries permitted until 2026-07-22 01:38:12.30773253 +0000 UTC m=+77.136980855 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-6f9lw" (UID: "26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1") : secret "cluster-monitoring-operator-tls" not found Jul 22 01:37:56.310074 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:56.310054 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/53cff89d-d019-4bd3-b3ea-1eea796beb99-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-s4qrb\" (UID: \"53cff89d-d019-4bd3-b3ea-1eea796beb99\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb" Jul 22 01:37:56.409024 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:56.408984 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:56.409197 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:56.409041 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-bmvq8\" (UID: \"cc79c9c1-844a-4780-9aa1-b433eeea8848\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8" Jul 22 01:37:56.409197 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:56.409118 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:56.409197 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:56.409162 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:37:56.409197 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:56.409129 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 01:37:56.409329 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:56.409216 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 01:37:56.409329 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:56.409170 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 01:37:56.409329 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:56.409235 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert podName:cc79c9c1-844a-4780-9aa1-b433eeea8848 nodeName:}" failed. No retries permitted until 2026-07-22 01:38:12.409217849 +0000 UTC m=+77.238466209 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-bmvq8" (UID: "cc79c9c1-844a-4780-9aa1-b433eeea8848") : secret "networking-console-plugin-cert" not found Jul 22 01:37:56.409329 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:56.409257 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs podName:e694641c-026f-4a5f-918b-6fedb0b96cdd nodeName:}" failed. No retries permitted until 2026-07-22 01:38:12.409243869 +0000 UTC m=+77.238492198 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs") pod "router-default-fb4497b6-psq7s" (UID: "e694641c-026f-4a5f-918b-6fedb0b96cdd") : secret "router-metrics-certs-default" not found Jul 22 01:37:56.409329 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:56.409270 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-rbac-proxy-cm podName:d6d3be9e-b32f-4f80-b49f-38854284252f nodeName:}" failed. No retries permitted until 2026-07-22 01:38:04.409264144 +0000 UTC m=+69.238512473 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gdxvd" (UID: "d6d3be9e-b32f-4f80-b49f-38854284252f") : configmap "kube-rbac-proxy" not found Jul 22 01:37:56.409329 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:56.409281 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle podName:e694641c-026f-4a5f-918b-6fedb0b96cdd nodeName:}" failed. No retries permitted until 2026-07-22 01:38:12.409275497 +0000 UTC m=+77.238523822 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle") pod "router-default-fb4497b6-psq7s" (UID: "e694641c-026f-4a5f-918b-6fedb0b96cdd") : configmap references non-existent config key: service-ca.crt Jul 22 01:37:56.409329 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:56.409293 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:37:56.409607 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:56.409419 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 01:37:56.409607 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:37:56.409446 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls podName:d6d3be9e-b32f-4f80-b49f-38854284252f nodeName:}" failed. No retries permitted until 2026-07-22 01:38:04.409439139 +0000 UTC m=+69.238687465 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gdxvd" (UID: "d6d3be9e-b32f-4f80-b49f-38854284252f") : secret "insights-runtime-extractor-tls" not found Jul 22 01:37:56.581112 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:56.581082 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ljfv8\"" Jul 22 01:37:56.588813 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:56.588783 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb" Jul 22 01:37:56.757719 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:56.757678 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb"] Jul 22 01:37:57.049458 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:57.049419 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb" event={"ID":"53cff89d-d019-4bd3-b3ea-1eea796beb99","Type":"ContainerStarted","Data":"7c4c5abfdc2ca785a5d9d2352856cff502f7eb83834768309eaf041db9c188ac"} Jul 22 01:37:57.925672 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:57.925632 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7gfnj" Jul 22 01:37:59.058673 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:59.058326 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb" event={"ID":"53cff89d-d019-4bd3-b3ea-1eea796beb99","Type":"ContainerStarted","Data":"1c971bd1066494b72d5c6752b598078d6d579d8621009b67fceabf64de31778c"} Jul 22 01:37:59.058673 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:37:59.058428 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb" event={"ID":"53cff89d-d019-4bd3-b3ea-1eea796beb99","Type":"ContainerStarted","Data":"37969d33ddcbbb9b8c868c8a6a016de7691e341c8ed2584449aab6ea9c0a2812"} Jul 22 01:38:01.257192 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:01.257145 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzv4z\" (UniqueName: \"kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z\") pod \"network-check-target-zrkr9\" (UID: \"4326c7c1-88b7-4ead-be08-a2a2e2497a64\") " pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:38:01.259694 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:01.259665 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzv4z\" (UniqueName: \"kubernetes.io/projected/4326c7c1-88b7-4ead-be08-a2a2e2497a64-kube-api-access-fzv4z\") pod \"network-check-target-zrkr9\" (UID: \"4326c7c1-88b7-4ead-be08-a2a2e2497a64\") " pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:38:01.358123 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:01.358079 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs\") pod \"network-metrics-daemon-878hc\" (UID: \"8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e\") " pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:38:01.358123 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:01.358126 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret\") pod \"global-pull-secret-syncer-tlbhv\" (UID: \"51b9bc4d-f596-4c94-9337-1428391e8aef\") " pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:38:01.360794 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:01.360769 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 01:38:01.360863 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:01.360781 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 01:38:01.368489 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:38:01.368456 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 22 01:38:01.368626 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:38:01.368538 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs podName:8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e nodeName:}" failed. No retries permitted until 2026-07-22 01:39:05.368517795 +0000 UTC m=+130.197766124 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs") pod "network-metrics-daemon-878hc" (UID: "8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e") : secret "metrics-daemon-secret" not found Jul 22 01:38:01.370751 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:01.370726 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51b9bc4d-f596-4c94-9337-1428391e8aef-original-pull-secret\") pod \"global-pull-secret-syncer-tlbhv\" (UID: \"51b9bc4d-f596-4c94-9337-1428391e8aef\") " pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:38:01.433277 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:01.433237 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tqzqp\"" Jul 22 01:38:01.440550 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:01.440513 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:38:01.570053 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:01.569995 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-s4qrb" podStartSLOduration=19.588924845 podStartE2EDuration="21.569976367s" totalImagesPullingTime="1.981051522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:37:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:59.088295856 +0000 UTC m=+63.917544205" watchObservedRunningTime="2026-07-22 01:38:01.569976367 +0000 UTC m=+66.399224713" Jul 22 01:38:01.570233 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:01.570163 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zrkr9"] Jul 22 01:38:01.573072 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:38:01.573041 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4326c7c1_88b7_4ead_be08_a2a2e2497a64.slice/crio-779ef23677ab98555ffcf93b91d648a8136e3046b61b4ab4d080de46620f3afc WatchSource:0}: Error finding container 779ef23677ab98555ffcf93b91d648a8136e3046b61b4ab4d080de46620f3afc: Status 404 returned error can't find the container with id 779ef23677ab98555ffcf93b91d648a8136e3046b61b4ab4d080de46620f3afc Jul 22 01:38:01.631436 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:01.631401 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tlbhv" Jul 22 01:38:01.761781 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:01.761622 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tlbhv"] Jul 22 01:38:01.764443 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:38:01.764410 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51b9bc4d_f596_4c94_9337_1428391e8aef.slice/crio-4ba629fe44af7002a784313bed4fc5ea0e7c02a55df375e1564cf2f7df14660d WatchSource:0}: Error finding container 4ba629fe44af7002a784313bed4fc5ea0e7c02a55df375e1564cf2f7df14660d: Status 404 returned error can't find the container with id 4ba629fe44af7002a784313bed4fc5ea0e7c02a55df375e1564cf2f7df14660d Jul 22 01:38:02.068942 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:02.068849 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zrkr9" event={"ID":"4326c7c1-88b7-4ead-be08-a2a2e2497a64","Type":"ContainerStarted","Data":"00c7b5630553d8355f37781e7b784ed7155e56a7d7efc3941d56df2af38d1909"} Jul 22 01:38:02.068942 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:02.068900 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zrkr9" event={"ID":"4326c7c1-88b7-4ead-be08-a2a2e2497a64","Type":"ContainerStarted","Data":"779ef23677ab98555ffcf93b91d648a8136e3046b61b4ab4d080de46620f3afc"} Jul 22 01:38:02.068942 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:02.068923 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:38:02.070045 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:02.070021 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tlbhv" event={"ID":"51b9bc4d-f596-4c94-9337-1428391e8aef","Type":"ContainerStarted","Data":"4ba629fe44af7002a784313bed4fc5ea0e7c02a55df375e1564cf2f7df14660d"} Jul 22 01:38:02.089984 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:02.089929 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zrkr9" podStartSLOduration=67.089913105 podStartE2EDuration="1m7.089913105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:36:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:38:02.088271311 +0000 UTC m=+66.917519658" watchObservedRunningTime="2026-07-22 01:38:02.089913105 +0000 UTC m=+66.919161452" Jul 22 01:38:02.973119 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:02.973081 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:38:02.976117 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:02.976083 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls\") pod \"image-registry-687855c694-q2wpf\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:38:03.073798 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:03.073765 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls\") pod \"dns-default-g2zdj\" (UID: \"cb80e90e-d908-490b-a9da-bb1d624d6615\") " pod="openshift-dns/dns-default-g2zdj" Jul 22 01:38:03.076489 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:03.076462 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb80e90e-d908-490b-a9da-bb1d624d6615-metrics-tls\") pod \"dns-default-g2zdj\" (UID: \"cb80e90e-d908-490b-a9da-bb1d624d6615\") " pod="openshift-dns/dns-default-g2zdj" Jul 22 01:38:03.183130 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:03.183100 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fnkgz\"" Jul 22 01:38:03.191180 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:03.191143 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:38:03.205915 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:03.205878 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fhl28\"" Jul 22 01:38:03.213325 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:03.213250 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g2zdj" Jul 22 01:38:03.388253 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:03.388218 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-687855c694-q2wpf"] Jul 22 01:38:03.394076 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:38:03.394039 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ea58868_1c24_48f4_8c3d_2427d0a6dbe2.slice/crio-c9b117bc326f6553a153227e7b791418165ea746a3f61b6354ea1e60cb13f81f WatchSource:0}: Error finding container c9b117bc326f6553a153227e7b791418165ea746a3f61b6354ea1e60cb13f81f: Status 404 returned error can't find the container with id c9b117bc326f6553a153227e7b791418165ea746a3f61b6354ea1e60cb13f81f Jul 22 01:38:03.415261 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:03.415225 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g2zdj"] Jul 22 01:38:03.419248 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:38:03.419212 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb80e90e_d908_490b_a9da_bb1d624d6615.slice/crio-a5d3e9eb428405b52846bbab7f50564175993a2428552496f3c20f3e5d1e52d7 WatchSource:0}: Error finding container a5d3e9eb428405b52846bbab7f50564175993a2428552496f3c20f3e5d1e52d7: Status 404 returned error can't find the container with id a5d3e9eb428405b52846bbab7f50564175993a2428552496f3c20f3e5d1e52d7 Jul 22 01:38:04.080309 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:04.079961 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-687855c694-q2wpf" event={"ID":"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2","Type":"ContainerStarted","Data":"16cc5e96ac4c715b0558a9fcc017877f53fe156cfa187e1fac60788e89329f01"} Jul 22 01:38:04.080309 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:04.080038 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-687855c694-q2wpf" event={"ID":"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2","Type":"ContainerStarted","Data":"c9b117bc326f6553a153227e7b791418165ea746a3f61b6354ea1e60cb13f81f"} Jul 22 01:38:04.080309 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:04.080073 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:38:04.082224 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:04.082160 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g2zdj" event={"ID":"cb80e90e-d908-490b-a9da-bb1d624d6615","Type":"ContainerStarted","Data":"a5d3e9eb428405b52846bbab7f50564175993a2428552496f3c20f3e5d1e52d7"} Jul 22 01:38:04.106154 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:04.106029 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-687855c694-q2wpf" podStartSLOduration=68.106004486 podStartE2EDuration="1m8.106004486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:36:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:38:04.104805102 +0000 UTC m=+68.934053451" watchObservedRunningTime="2026-07-22 01:38:04.106004486 +0000 UTC m=+68.935252836" Jul 22 01:38:04.489240 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:04.489149 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:38:04.489240 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:04.489215 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:38:04.489633 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:38:04.489335 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 01:38:04.489633 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:38:04.489413 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls podName:d6d3be9e-b32f-4f80-b49f-38854284252f nodeName:}" failed. No retries permitted until 2026-07-22 01:38:20.489397731 +0000 UTC m=+85.318646057 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gdxvd" (UID: "d6d3be9e-b32f-4f80-b49f-38854284252f") : secret "insights-runtime-extractor-tls" not found Jul 22 01:38:04.489845 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:04.489825 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d6d3be9e-b32f-4f80-b49f-38854284252f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:38:04.620207 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:04.620176 2559 scope.go:117] "RemoveContainer" containerID="39c66b4c0fd7dbc19cb4ef10b37261e3a924a60a43885b2df72b0fe461ff2cc9" Jul 22 01:38:07.095523 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:07.095486 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 01:38:07.096033 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:07.095617 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" event={"ID":"546e33d1-6966-445c-9d8d-d10cf6631943","Type":"ContainerStarted","Data":"2d8b57aac356ac1b09fa7a90bcffe4feed8e37769697c68aa642237f674945af"} Jul 22 01:38:07.096033 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:07.095986 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" Jul 22 01:38:07.097438 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:07.097401 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tlbhv" event={"ID":"51b9bc4d-f596-4c94-9337-1428391e8aef","Type":"ContainerStarted","Data":"2c7240d1d48b9f7d313a871c69ceffae73a2c04de5ebeace1076e0593501d8ba"} Jul 22 01:38:07.099208 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:07.099184 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g2zdj" event={"ID":"cb80e90e-d908-490b-a9da-bb1d624d6615","Type":"ContainerStarted","Data":"dab0fdab4c2f90384d0b09cabf6415cc405af6857bdcdd2a5d091586129eec1a"} Jul 22 01:38:07.099339 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:07.099230 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g2zdj" event={"ID":"cb80e90e-d908-490b-a9da-bb1d624d6615","Type":"ContainerStarted","Data":"4eac76a707e58de72c002f42cfadef3b73d054e92b0dc0a759cb7ee7770e8151"} Jul 22 01:38:07.099339 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:07.099327 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-g2zdj" Jul 22 01:38:07.113971 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:07.113919 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" podStartSLOduration=20.732232649 podStartE2EDuration="27.113903032s" totalImagesPullingTime="6.381670383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:37:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:38:07.113091611 +0000 UTC m=+71.942339961" watchObservedRunningTime="2026-07-22 01:38:07.113903032 +0000 UTC m=+71.943151417" Jul 22 01:38:07.129316 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:07.129262 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tlbhv" podStartSLOduration=66.442822564 podStartE2EDuration="1m11.12924275s" totalImagesPullingTime="4.686420186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:38:07.127966374 +0000 UTC m=+71.957214715" watchObservedRunningTime="2026-07-22 01:38:07.12924275 +0000 UTC m=+71.958491097" Jul 22 01:38:07.145041 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:07.144988 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-g2zdj" podStartSLOduration=33.12931994 podStartE2EDuration="36.144971727s" totalImagesPullingTime="3.015651787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:37:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:38:07.144021722 +0000 UTC m=+71.973270071" watchObservedRunningTime="2026-07-22 01:38:07.144971727 +0000 UTC m=+71.974220103" Jul 22 01:38:07.200668 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:07.200636 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-z6wbm" Jul 22 01:38:12.362377 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:12.362304 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-6f9lw\" (UID: \"26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" Jul 22 01:38:12.364782 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:12.364747 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-6f9lw\" (UID: \"26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" Jul 22 01:38:12.463538 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:12.463493 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:38:12.463719 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:12.463551 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-bmvq8\" (UID: \"cc79c9c1-844a-4780-9aa1-b433eeea8848\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8" Jul 22 01:38:12.463719 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:12.463680 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:38:12.464295 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:12.464255 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e694641c-026f-4a5f-918b-6fedb0b96cdd-service-ca-bundle\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:38:12.465946 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:12.465911 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cc79c9c1-844a-4780-9aa1-b433eeea8848-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-bmvq8\" (UID: \"cc79c9c1-844a-4780-9aa1-b433eeea8848\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8" Jul 22 01:38:12.466072 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:12.465959 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e694641c-026f-4a5f-918b-6fedb0b96cdd-metrics-certs\") pod \"router-default-fb4497b6-psq7s\" (UID: \"e694641c-026f-4a5f-918b-6fedb0b96cdd\") " pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:38:12.509997 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:12.509964 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6jxxw\"" Jul 22 01:38:12.518795 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:12.518767 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" Jul 22 01:38:12.567728 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:12.567688 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qs5rw\"" Jul 22 01:38:12.588820 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:12.587541 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:38:12.614178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:12.614104 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qn4sz\"" Jul 22 01:38:12.621370 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:12.621262 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8" Jul 22 01:38:12.688531 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:12.688496 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw"] Jul 22 01:38:12.693857 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:38:12.693631 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26c374ff_c41f_44fe_b7f4_6d9b92b6bcb1.slice/crio-6680e52d8c64658023e388957b40dcc2b30984c4735d7a41b139495de8a79458 WatchSource:0}: Error finding container 6680e52d8c64658023e388957b40dcc2b30984c4735d7a41b139495de8a79458: Status 404 returned error can't find the container with id 6680e52d8c64658023e388957b40dcc2b30984c4735d7a41b139495de8a79458 Jul 22 01:38:12.753999 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:12.753971 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-fb4497b6-psq7s"] Jul 22 01:38:12.756897 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:38:12.756860 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode694641c_026f_4a5f_918b_6fedb0b96cdd.slice/crio-fa6dbb8dadb4668d01b2e58303bfdb77c6ced574b4b09692423c43779716fb40 WatchSource:0}: Error finding container fa6dbb8dadb4668d01b2e58303bfdb77c6ced574b4b09692423c43779716fb40: Status 404 returned error can't find the container with id fa6dbb8dadb4668d01b2e58303bfdb77c6ced574b4b09692423c43779716fb40 Jul 22 01:38:12.772195 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:12.772164 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8"] Jul 22 01:38:12.775228 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:38:12.775195 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc79c9c1_844a_4780_9aa1_b433eeea8848.slice/crio-2be3efde338095089582f26e1efe9561ac00615fe17aa94713485af837da2174 WatchSource:0}: Error finding container 2be3efde338095089582f26e1efe9561ac00615fe17aa94713485af837da2174: Status 404 returned error can't find the container with id 2be3efde338095089582f26e1efe9561ac00615fe17aa94713485af837da2174 Jul 22 01:38:13.120699 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:13.120661 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8" event={"ID":"cc79c9c1-844a-4780-9aa1-b433eeea8848","Type":"ContainerStarted","Data":"2be3efde338095089582f26e1efe9561ac00615fe17aa94713485af837da2174"} Jul 22 01:38:13.121727 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:13.121700 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" event={"ID":"26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1","Type":"ContainerStarted","Data":"6680e52d8c64658023e388957b40dcc2b30984c4735d7a41b139495de8a79458"} Jul 22 01:38:13.123086 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:13.123061 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-fb4497b6-psq7s" event={"ID":"e694641c-026f-4a5f-918b-6fedb0b96cdd","Type":"ContainerStarted","Data":"356aff50488d351c571e6278ecbad211e5991108e1ef5bc297890dabd08348e0"} Jul 22 01:38:13.123171 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:13.123096 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-fb4497b6-psq7s" event={"ID":"e694641c-026f-4a5f-918b-6fedb0b96cdd","Type":"ContainerStarted","Data":"fa6dbb8dadb4668d01b2e58303bfdb77c6ced574b4b09692423c43779716fb40"} Jul 22 01:38:13.146442 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:13.146371 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-fb4497b6-psq7s" podStartSLOduration=33.146334355 podStartE2EDuration="33.146334355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:37:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:38:13.14574858 +0000 UTC m=+77.974996927" watchObservedRunningTime="2026-07-22 01:38:13.146334355 +0000 UTC m=+77.975582737" Jul 22 01:38:13.588610 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:13.588565 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:38:13.591298 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:13.591273 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:38:14.128723 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:14.128685 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:38:14.130531 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:14.130499 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-fb4497b6-psq7s" Jul 22 01:38:15.134285 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:15.134235 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" event={"ID":"26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1","Type":"ContainerStarted","Data":"0b334bd228b8f9b09113dc60ea489d5b2aeb9363613e847475d8fa0bf8d5035c"} Jul 22 01:38:15.135986 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:15.135958 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8" event={"ID":"cc79c9c1-844a-4780-9aa1-b433eeea8848","Type":"ContainerStarted","Data":"ff6f9ebff5c5abd648dfb30fd5c58115b61035f88d81f42e89a3013c2895be31"} Jul 22 01:38:15.152463 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:15.152405 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-6f9lw" podStartSLOduration=33.096935979 podStartE2EDuration="35.152388029s" totalImagesPullingTime="2.05545205s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:37:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:38:15.151478553 +0000 UTC m=+79.980726902" watchObservedRunningTime="2026-07-22 01:38:15.152388029 +0000 UTC m=+79.981636425" Jul 22 01:38:15.173504 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:15.173438 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-bmvq8" podStartSLOduration=33.20378536 podStartE2EDuration="35.173416748s" totalImagesPullingTime="1.969631388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:37:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:38:15.173087156 +0000 UTC m=+80.002335515" watchObservedRunningTime="2026-07-22 01:38:15.173416748 +0000 UTC m=+80.002665129" Jul 22 01:38:17.105031 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:17.104999 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-g2zdj" Jul 22 01:38:20.525589 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:20.525544 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:38:20.527928 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:20.527899 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d6d3be9e-b32f-4f80-b49f-38854284252f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gdxvd\" (UID: \"d6d3be9e-b32f-4f80-b49f-38854284252f\") " pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:38:20.830530 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:20.830496 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-l77sq\"" Jul 22 01:38:20.838404 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:20.838379 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gdxvd" Jul 22 01:38:21.008262 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:21.008223 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gdxvd"] Jul 22 01:38:21.011154 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:38:21.011126 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6d3be9e_b32f_4f80_b49f_38854284252f.slice/crio-e295116a4c439a9f04ea1797e48929841782b0a85aaeb921f9435cec293b0a60 WatchSource:0}: Error finding container e295116a4c439a9f04ea1797e48929841782b0a85aaeb921f9435cec293b0a60: Status 404 returned error can't find the container with id e295116a4c439a9f04ea1797e48929841782b0a85aaeb921f9435cec293b0a60 Jul 22 01:38:21.159277 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:21.159184 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gdxvd" event={"ID":"d6d3be9e-b32f-4f80-b49f-38854284252f","Type":"ContainerStarted","Data":"e295116a4c439a9f04ea1797e48929841782b0a85aaeb921f9435cec293b0a60"} Jul 22 01:38:23.512533 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.510611 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c9fc84455-szd78"] Jul 22 01:38:23.524027 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.523989 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c9fc84455-szd78"] Jul 22 01:38:23.524238 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.524144 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.527776 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.527748 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 22 01:38:23.528059 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.527786 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 01:38:23.528184 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.527864 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 01:38:23.528184 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.528041 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 01:38:23.528993 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.528970 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kl75w\"" Jul 22 01:38:23.529194 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.529046 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 01:38:23.529194 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.529114 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 22 01:38:23.529582 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.529560 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 01:38:23.533270 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.532954 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 22 01:38:23.552803 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.552750 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-trusted-ca-bundle\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.553018 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.552819 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-console-config\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.553018 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.552853 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/346ef971-b332-4cd5-801c-28a47c1fa213-console-oauth-config\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.553018 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.552891 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lmmb\" (UniqueName: \"kubernetes.io/projected/346ef971-b332-4cd5-801c-28a47c1fa213-kube-api-access-6lmmb\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.553018 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.552936 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-oauth-serving-cert\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.553018 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.552971 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-service-ca\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.553304 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.553053 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/346ef971-b332-4cd5-801c-28a47c1fa213-console-serving-cert\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.653553 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.653517 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/346ef971-b332-4cd5-801c-28a47c1fa213-console-serving-cert\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.653751 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.653580 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-trusted-ca-bundle\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.653751 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.653636 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-console-config\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.653751 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.653660 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/346ef971-b332-4cd5-801c-28a47c1fa213-console-oauth-config\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.653751 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.653694 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lmmb\" (UniqueName: \"kubernetes.io/projected/346ef971-b332-4cd5-801c-28a47c1fa213-kube-api-access-6lmmb\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.653751 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.653738 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-oauth-serving-cert\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.654016 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.653773 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-service-ca\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.654490 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.654458 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-console-config\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.654622 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.654499 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-service-ca\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.654622 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.654580 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-oauth-serving-cert\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.654622 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.654608 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-trusted-ca-bundle\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.656496 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.656475 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/346ef971-b332-4cd5-801c-28a47c1fa213-console-serving-cert\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.656610 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.656585 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/346ef971-b332-4cd5-801c-28a47c1fa213-console-oauth-config\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.663058 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.663024 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lmmb\" (UniqueName: \"kubernetes.io/projected/346ef971-b332-4cd5-801c-28a47c1fa213-kube-api-access-6lmmb\") pod \"console-5c9fc84455-szd78\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:23.837878 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:23.837843 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:24.402594 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.402553 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq"] Jul 22 01:38:24.415268 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.415234 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq"] Jul 22 01:38:24.415487 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.415310 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq" Jul 22 01:38:24.417710 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.417682 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 22 01:38:24.417904 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.417804 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 22 01:38:24.417904 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.417804 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 01:38:24.418023 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.417960 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-8t2qt\"" Jul 22 01:38:24.461549 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.461507 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f2bab6b-76d2-41b0-a2d9-8f8bfc824009-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-mbwwq\" (UID: \"2f2bab6b-76d2-41b0-a2d9-8f8bfc824009\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq" Jul 22 01:38:24.461549 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.461550 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2f2bab6b-76d2-41b0-a2d9-8f8bfc824009-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-mbwwq\" (UID: \"2f2bab6b-76d2-41b0-a2d9-8f8bfc824009\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq" Jul 22 01:38:24.461776 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.461686 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f2bab6b-76d2-41b0-a2d9-8f8bfc824009-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-mbwwq\" (UID: \"2f2bab6b-76d2-41b0-a2d9-8f8bfc824009\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq" Jul 22 01:38:24.461776 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.461733 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfz8v\" (UniqueName: \"kubernetes.io/projected/2f2bab6b-76d2-41b0-a2d9-8f8bfc824009-kube-api-access-xfz8v\") pod \"prometheus-operator-5945b5cfd-mbwwq\" (UID: \"2f2bab6b-76d2-41b0-a2d9-8f8bfc824009\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq" Jul 22 01:38:24.562831 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.562797 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfz8v\" (UniqueName: \"kubernetes.io/projected/2f2bab6b-76d2-41b0-a2d9-8f8bfc824009-kube-api-access-xfz8v\") pod \"prometheus-operator-5945b5cfd-mbwwq\" (UID: \"2f2bab6b-76d2-41b0-a2d9-8f8bfc824009\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq" Jul 22 01:38:24.563321 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.562863 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f2bab6b-76d2-41b0-a2d9-8f8bfc824009-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-mbwwq\" (UID: \"2f2bab6b-76d2-41b0-a2d9-8f8bfc824009\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq" Jul 22 01:38:24.563321 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.562892 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2f2bab6b-76d2-41b0-a2d9-8f8bfc824009-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-mbwwq\" (UID: \"2f2bab6b-76d2-41b0-a2d9-8f8bfc824009\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq" Jul 22 01:38:24.563321 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.562983 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f2bab6b-76d2-41b0-a2d9-8f8bfc824009-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-mbwwq\" (UID: \"2f2bab6b-76d2-41b0-a2d9-8f8bfc824009\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq" Jul 22 01:38:24.563676 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.563649 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2f2bab6b-76d2-41b0-a2d9-8f8bfc824009-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-mbwwq\" (UID: \"2f2bab6b-76d2-41b0-a2d9-8f8bfc824009\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq" Jul 22 01:38:24.565779 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.565751 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2f2bab6b-76d2-41b0-a2d9-8f8bfc824009-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-mbwwq\" (UID: \"2f2bab6b-76d2-41b0-a2d9-8f8bfc824009\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq" Jul 22 01:38:24.565903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.565802 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2f2bab6b-76d2-41b0-a2d9-8f8bfc824009-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-mbwwq\" (UID: \"2f2bab6b-76d2-41b0-a2d9-8f8bfc824009\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq" Jul 22 01:38:24.571845 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.571818 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfz8v\" (UniqueName: \"kubernetes.io/projected/2f2bab6b-76d2-41b0-a2d9-8f8bfc824009-kube-api-access-xfz8v\") pod \"prometheus-operator-5945b5cfd-mbwwq\" (UID: \"2f2bab6b-76d2-41b0-a2d9-8f8bfc824009\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq" Jul 22 01:38:24.727610 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:24.727527 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq" Jul 22 01:38:25.092431 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:25.092393 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:38:26.022705 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:26.022678 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c9fc84455-szd78"] Jul 22 01:38:26.025766 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:38:26.025732 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod346ef971_b332_4cd5_801c_28a47c1fa213.slice/crio-ed690fff4df4606cafed49a0d22f5c616d98b535259ba706a13ec7a8a3288cfa WatchSource:0}: Error finding container ed690fff4df4606cafed49a0d22f5c616d98b535259ba706a13ec7a8a3288cfa: Status 404 returned error can't find the container with id ed690fff4df4606cafed49a0d22f5c616d98b535259ba706a13ec7a8a3288cfa Jul 22 01:38:26.047897 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:26.047825 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq"] Jul 22 01:38:26.052297 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:38:26.052225 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f2bab6b_76d2_41b0_a2d9_8f8bfc824009.slice/crio-89d9141175f1dea1b7a7e28cbc515e8479bb532619004fd2a0138606d96a8e81 WatchSource:0}: Error finding container 89d9141175f1dea1b7a7e28cbc515e8479bb532619004fd2a0138606d96a8e81: Status 404 returned error can't find the container with id 89d9141175f1dea1b7a7e28cbc515e8479bb532619004fd2a0138606d96a8e81 Jul 22 01:38:26.180810 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:26.180723 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gdxvd" event={"ID":"d6d3be9e-b32f-4f80-b49f-38854284252f","Type":"ContainerStarted","Data":"5fe0ca9dbb22f83e065d69887cf1d69f0d6dcaac2da3a83e697da83940299867"} Jul 22 01:38:26.181951 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:26.181903 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq" event={"ID":"2f2bab6b-76d2-41b0-a2d9-8f8bfc824009","Type":"ContainerStarted","Data":"89d9141175f1dea1b7a7e28cbc515e8479bb532619004fd2a0138606d96a8e81"} Jul 22 01:38:26.183171 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:26.183146 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c9fc84455-szd78" event={"ID":"346ef971-b332-4cd5-801c-28a47c1fa213","Type":"ContainerStarted","Data":"ed690fff4df4606cafed49a0d22f5c616d98b535259ba706a13ec7a8a3288cfa"} Jul 22 01:38:32.211715 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:32.211672 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gdxvd" event={"ID":"d6d3be9e-b32f-4f80-b49f-38854284252f","Type":"ContainerStarted","Data":"4fdc76e6ddbc0ee6ce1583abe57f83cd271e873de1af2d1926c301bb05915498"} Jul 22 01:38:32.213542 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:32.213509 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq" event={"ID":"2f2bab6b-76d2-41b0-a2d9-8f8bfc824009","Type":"ContainerStarted","Data":"e0745371d3bd6c7e58b99bd04b38d9961d6fcb4d840726d2db5e6a8edb546100"} Jul 22 01:38:32.213702 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:32.213567 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq" event={"ID":"2f2bab6b-76d2-41b0-a2d9-8f8bfc824009","Type":"ContainerStarted","Data":"34860a2a9f7ec29a39f01dcd630bb5284bab66853a4388504dc22a0be3c79903"} Jul 22 01:38:32.215013 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:32.214983 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c9fc84455-szd78" event={"ID":"346ef971-b332-4cd5-801c-28a47c1fa213","Type":"ContainerStarted","Data":"a3b87065d9e733574322dbb26e6f7acd75713dd1993878d9bf63be4fb9cfc0ff"} Jul 22 01:38:32.238537 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:32.238475 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-mbwwq" podStartSLOduration=2.989116558 podStartE2EDuration="8.238452139s" totalImagesPullingTime="5.249335581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:38:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:38:32.235691795 +0000 UTC m=+97.064940142" watchObservedRunningTime="2026-07-22 01:38:32.238452139 +0000 UTC m=+97.067700488" Jul 22 01:38:32.261047 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:32.260988 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c9fc84455-szd78" podStartSLOduration=3.985120098 podStartE2EDuration="9.260969752s" totalImagesPullingTime="5.275849654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:38:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:38:32.259191643 +0000 UTC m=+97.088439993" watchObservedRunningTime="2026-07-22 01:38:32.260969752 +0000 UTC m=+97.090218099" Jul 22 01:38:33.076810 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:33.076778 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zrkr9" Jul 22 01:38:33.838863 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:33.838822 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:33.839307 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:33.838883 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:33.844549 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:33.844517 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:34.111265 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.111175 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-h8mll"] Jul 22 01:38:34.128465 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.128024 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.131227 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.131197 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 22 01:38:34.131466 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.131197 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 22 01:38:34.131559 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.131533 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-dthth\"" Jul 22 01:38:34.132014 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.131871 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 22 01:38:34.132014 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.131980 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-h8mll"] Jul 22 01:38:34.134906 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.134874 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-c9qzd"] Jul 22 01:38:34.140289 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.140257 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.144734 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.144691 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 01:38:34.144915 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.144696 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 01:38:34.145606 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.145581 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-npxl9\"" Jul 22 01:38:34.145729 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.145680 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 01:38:34.230222 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.230192 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:38:34.255942 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.255908 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a58cd2c5-b36e-4ad1-aed7-f5b378012004-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-h8mll\" (UID: \"a58cd2c5-b36e-4ad1-aed7-f5b378012004\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.256144 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.255971 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1acf37c9-d8f6-4dcf-b867-580ccc196908-metrics-client-ca\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.256144 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.256002 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a58cd2c5-b36e-4ad1-aed7-f5b378012004-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-h8mll\" (UID: \"a58cd2c5-b36e-4ad1-aed7-f5b378012004\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.256144 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.256027 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1acf37c9-d8f6-4dcf-b867-580ccc196908-sys\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.256144 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.256047 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1acf37c9-d8f6-4dcf-b867-580ccc196908-root\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.256144 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.256077 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1acf37c9-d8f6-4dcf-b867-580ccc196908-node-exporter-wtmp\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.256144 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.256097 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a58cd2c5-b36e-4ad1-aed7-f5b378012004-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-h8mll\" (UID: \"a58cd2c5-b36e-4ad1-aed7-f5b378012004\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.256144 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.256121 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1acf37c9-d8f6-4dcf-b867-580ccc196908-node-exporter-tls\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.256144 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.256143 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1acf37c9-d8f6-4dcf-b867-580ccc196908-node-exporter-accelerators-collector-config\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.256569 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.256191 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a58cd2c5-b36e-4ad1-aed7-f5b378012004-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-h8mll\" (UID: \"a58cd2c5-b36e-4ad1-aed7-f5b378012004\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.256569 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.256282 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-894fn\" (UniqueName: \"kubernetes.io/projected/a58cd2c5-b36e-4ad1-aed7-f5b378012004-kube-api-access-894fn\") pod \"kube-state-metrics-56c5594674-h8mll\" (UID: \"a58cd2c5-b36e-4ad1-aed7-f5b378012004\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.256569 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.256549 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5p26\" (UniqueName: \"kubernetes.io/projected/1acf37c9-d8f6-4dcf-b867-580ccc196908-kube-api-access-z5p26\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.256720 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.256604 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1acf37c9-d8f6-4dcf-b867-580ccc196908-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.256720 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.256640 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a58cd2c5-b36e-4ad1-aed7-f5b378012004-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-h8mll\" (UID: \"a58cd2c5-b36e-4ad1-aed7-f5b378012004\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.256720 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.256695 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1acf37c9-d8f6-4dcf-b867-580ccc196908-node-exporter-textfile\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.358373 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.358155 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1acf37c9-d8f6-4dcf-b867-580ccc196908-node-exporter-textfile\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.358373 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.358246 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a58cd2c5-b36e-4ad1-aed7-f5b378012004-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-h8mll\" (UID: \"a58cd2c5-b36e-4ad1-aed7-f5b378012004\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.359084 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.358984 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1acf37c9-d8f6-4dcf-b867-580ccc196908-node-exporter-textfile\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.359084 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.358993 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1acf37c9-d8f6-4dcf-b867-580ccc196908-metrics-client-ca\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.359084 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.359050 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a58cd2c5-b36e-4ad1-aed7-f5b378012004-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-h8mll\" (UID: \"a58cd2c5-b36e-4ad1-aed7-f5b378012004\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.359231 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.359103 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1acf37c9-d8f6-4dcf-b867-580ccc196908-sys\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.359231 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.359127 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1acf37c9-d8f6-4dcf-b867-580ccc196908-root\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.359231 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.359165 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1acf37c9-d8f6-4dcf-b867-580ccc196908-node-exporter-wtmp\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.359231 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.359189 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a58cd2c5-b36e-4ad1-aed7-f5b378012004-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-h8mll\" (UID: \"a58cd2c5-b36e-4ad1-aed7-f5b378012004\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.359231 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.359220 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1acf37c9-d8f6-4dcf-b867-580ccc196908-node-exporter-tls\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.359505 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.359249 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1acf37c9-d8f6-4dcf-b867-580ccc196908-node-exporter-accelerators-collector-config\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.359505 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.359327 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a58cd2c5-b36e-4ad1-aed7-f5b378012004-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-h8mll\" (UID: \"a58cd2c5-b36e-4ad1-aed7-f5b378012004\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.359505 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.359388 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-894fn\" (UniqueName: \"kubernetes.io/projected/a58cd2c5-b36e-4ad1-aed7-f5b378012004-kube-api-access-894fn\") pod \"kube-state-metrics-56c5594674-h8mll\" (UID: \"a58cd2c5-b36e-4ad1-aed7-f5b378012004\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.359505 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.359453 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5p26\" (UniqueName: \"kubernetes.io/projected/1acf37c9-d8f6-4dcf-b867-580ccc196908-kube-api-access-z5p26\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.359505 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.359482 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1acf37c9-d8f6-4dcf-b867-580ccc196908-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.359746 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.359508 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a58cd2c5-b36e-4ad1-aed7-f5b378012004-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-h8mll\" (UID: \"a58cd2c5-b36e-4ad1-aed7-f5b378012004\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.360687 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.359804 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1acf37c9-d8f6-4dcf-b867-580ccc196908-metrics-client-ca\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.360687 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.359863 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1acf37c9-d8f6-4dcf-b867-580ccc196908-root\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.360687 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.359923 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1acf37c9-d8f6-4dcf-b867-580ccc196908-sys\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.360687 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.360022 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a58cd2c5-b36e-4ad1-aed7-f5b378012004-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-h8mll\" (UID: \"a58cd2c5-b36e-4ad1-aed7-f5b378012004\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.360687 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.360176 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1acf37c9-d8f6-4dcf-b867-580ccc196908-node-exporter-accelerators-collector-config\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.360687 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:38:34.360285 2559 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 22 01:38:34.360687 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.360285 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a58cd2c5-b36e-4ad1-aed7-f5b378012004-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-h8mll\" (UID: \"a58cd2c5-b36e-4ad1-aed7-f5b378012004\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.360687 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:38:34.360387 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1acf37c9-d8f6-4dcf-b867-580ccc196908-node-exporter-tls podName:1acf37c9-d8f6-4dcf-b867-580ccc196908 nodeName:}" failed. No retries permitted until 2026-07-22 01:38:34.860336868 +0000 UTC m=+99.689585200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/1acf37c9-d8f6-4dcf-b867-580ccc196908-node-exporter-tls") pod "node-exporter-c9qzd" (UID: "1acf37c9-d8f6-4dcf-b867-580ccc196908") : secret "node-exporter-tls" not found Jul 22 01:38:34.360687 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.360557 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1acf37c9-d8f6-4dcf-b867-580ccc196908-node-exporter-wtmp\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.360687 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.360623 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a58cd2c5-b36e-4ad1-aed7-f5b378012004-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-h8mll\" (UID: \"a58cd2c5-b36e-4ad1-aed7-f5b378012004\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.363299 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.363243 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a58cd2c5-b36e-4ad1-aed7-f5b378012004-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-h8mll\" (UID: \"a58cd2c5-b36e-4ad1-aed7-f5b378012004\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.363431 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.363309 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1acf37c9-d8f6-4dcf-b867-580ccc196908-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.363909 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.363887 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a58cd2c5-b36e-4ad1-aed7-f5b378012004-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-h8mll\" (UID: \"a58cd2c5-b36e-4ad1-aed7-f5b378012004\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.373790 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.373725 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-894fn\" (UniqueName: \"kubernetes.io/projected/a58cd2c5-b36e-4ad1-aed7-f5b378012004-kube-api-access-894fn\") pod \"kube-state-metrics-56c5594674-h8mll\" (UID: \"a58cd2c5-b36e-4ad1-aed7-f5b378012004\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.376148 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.376088 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5p26\" (UniqueName: \"kubernetes.io/projected/1acf37c9-d8f6-4dcf-b867-580ccc196908-kube-api-access-z5p26\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.442178 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.442140 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" Jul 22 01:38:34.865959 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.865921 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1acf37c9-d8f6-4dcf-b867-580ccc196908-node-exporter-tls\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:34.868234 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:34.868201 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1acf37c9-d8f6-4dcf-b867-580ccc196908-node-exporter-tls\") pod \"node-exporter-c9qzd\" (UID: \"1acf37c9-d8f6-4dcf-b867-580ccc196908\") " pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:35.052417 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:35.052391 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c9qzd" Jul 22 01:38:35.064262 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:38:35.064233 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1acf37c9_d8f6_4dcf_b867_580ccc196908.slice/crio-e95a3defae67cd766e3cde5940f60d11fea31084a85052e736436249ba587a98 WatchSource:0}: Error finding container e95a3defae67cd766e3cde5940f60d11fea31084a85052e736436249ba587a98: Status 404 returned error can't find the container with id e95a3defae67cd766e3cde5940f60d11fea31084a85052e736436249ba587a98 Jul 22 01:38:35.065165 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:35.065128 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-h8mll"] Jul 22 01:38:35.067439 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:38:35.067414 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda58cd2c5_b36e_4ad1_aed7_f5b378012004.slice/crio-cdd2bfc4402e0ccfb6602fddcc6455b640de299b03c91a2a8839bd735590df4f WatchSource:0}: Error finding container cdd2bfc4402e0ccfb6602fddcc6455b640de299b03c91a2a8839bd735590df4f: Status 404 returned error can't find the container with id cdd2bfc4402e0ccfb6602fddcc6455b640de299b03c91a2a8839bd735590df4f Jul 22 01:38:35.231012 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:35.230974 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gdxvd" event={"ID":"d6d3be9e-b32f-4f80-b49f-38854284252f","Type":"ContainerStarted","Data":"443a6865ad0885eebf95039d26eb9665d1735b56776628ab8048e8519008fca9"} Jul 22 01:38:35.232265 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:35.232240 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" event={"ID":"a58cd2c5-b36e-4ad1-aed7-f5b378012004","Type":"ContainerStarted","Data":"cdd2bfc4402e0ccfb6602fddcc6455b640de299b03c91a2a8839bd735590df4f"} Jul 22 01:38:35.233399 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:35.233377 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c9qzd" event={"ID":"1acf37c9-d8f6-4dcf-b867-580ccc196908","Type":"ContainerStarted","Data":"e95a3defae67cd766e3cde5940f60d11fea31084a85052e736436249ba587a98"} Jul 22 01:38:35.256639 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:35.256591 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gdxvd" podStartSLOduration=33.471624057 podStartE2EDuration="47.256574756s" totalImagesPullingTime="13.784950699s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:37:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:38:35.255940725 +0000 UTC m=+100.085189075" watchObservedRunningTime="2026-07-22 01:38:35.256574756 +0000 UTC m=+100.085823104" Jul 22 01:38:36.259822 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:36.259784 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-687855c694-q2wpf"] Jul 22 01:38:37.246477 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:37.246384 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" event={"ID":"a58cd2c5-b36e-4ad1-aed7-f5b378012004","Type":"ContainerStarted","Data":"5bd151978d1ee2c7f3c6af287c6dbdb3819298476ef7e03e91cf38780d9b9a80"} Jul 22 01:38:37.246477 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:37.246457 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" event={"ID":"a58cd2c5-b36e-4ad1-aed7-f5b378012004","Type":"ContainerStarted","Data":"67b7757a17f46e9bf4ce7cd988b4c77978cd0560eb89b0d6ba887895a5e3e9d3"} Jul 22 01:38:37.246688 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:37.246487 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" event={"ID":"a58cd2c5-b36e-4ad1-aed7-f5b378012004","Type":"ContainerStarted","Data":"486700334a40e9152ac71133cce8c18aa6f386b62b44d7a0db921ffd49dc96b9"} Jul 22 01:38:37.247903 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:37.247876 2559 generic.go:356] "Generic (PLEG): container finished" podID="1acf37c9-d8f6-4dcf-b867-580ccc196908" containerID="6a9bb3ef7a0ef63119da632616c1c62233681b9f746375a04f6090b753198047" exitCode=0 Jul 22 01:38:37.248038 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:37.247960 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c9qzd" event={"ID":"1acf37c9-d8f6-4dcf-b867-580ccc196908","Type":"ContainerDied","Data":"6a9bb3ef7a0ef63119da632616c1c62233681b9f746375a04f6090b753198047"} Jul 22 01:38:37.269850 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:37.269790 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-h8mll" podStartSLOduration=1.6233196319999998 podStartE2EDuration="3.269772664s" totalImagesPullingTime="1.646453032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:38:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:38:37.267645969 +0000 UTC m=+102.096894316" watchObservedRunningTime="2026-07-22 01:38:37.269772664 +0000 UTC m=+102.099021012" Jul 22 01:38:38.253805 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:38.253766 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c9qzd" event={"ID":"1acf37c9-d8f6-4dcf-b867-580ccc196908","Type":"ContainerStarted","Data":"92963c46e1dac0eea2dff0fe6f8416b7e7c859628fd969e1369d311e5bc78071"} Jul 22 01:38:38.254015 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:38.253824 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c9qzd" event={"ID":"1acf37c9-d8f6-4dcf-b867-580ccc196908","Type":"ContainerStarted","Data":"420166497bad542c7747de3083d03a156229c15e2532a3acd79a9a674dc35285"} Jul 22 01:38:38.287281 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:38.287214 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-c9qzd" podStartSLOduration=2.956030994 podStartE2EDuration="4.287197768s" totalImagesPullingTime="1.25339109s" totalInitContainerRuntime="77.775684ms" isStatefulPod=true podCreationTimestamp="2026-07-22 01:38:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:38:38.285412808 +0000 UTC m=+103.114661159" watchObservedRunningTime="2026-07-22 01:38:38.287197768 +0000 UTC m=+103.116446116" Jul 22 01:38:40.147257 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.147215 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 22 01:38:40.150637 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.150608 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.153695 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.153665 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-8tq4n\"" Jul 22 01:38:40.153867 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.153730 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 22 01:38:40.153867 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.153738 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 22 01:38:40.154179 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.154160 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 22 01:38:40.154766 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.154740 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 22 01:38:40.154879 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.154795 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 22 01:38:40.154941 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.154917 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 22 01:38:40.154992 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.154952 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9i1u2ip0sueb9\"" Jul 22 01:38:40.155093 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.155076 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 22 01:38:40.155166 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.155115 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 22 01:38:40.155166 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.155125 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 22 01:38:40.155661 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.155624 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 22 01:38:40.155661 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.155625 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 22 01:38:40.156884 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.156584 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 22 01:38:40.156884 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.156802 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 22 01:38:40.158714 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.158503 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 22 01:38:40.160358 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.160331 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 22 01:38:40.174762 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.174719 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 22 01:38:40.217605 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.217566 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-config\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.217605 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.217604 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.217836 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.217627 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5546c5e-1ba0-4c67-a465-8f8e612129a1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.217836 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.217659 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.217836 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.217712 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.217836 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.217732 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-web-config\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.217836 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.217753 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.217836 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.217771 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.217836 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.217795 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.217836 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.217811 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28s2w\" (UniqueName: \"kubernetes.io/projected/f5546c5e-1ba0-4c67-a465-8f8e612129a1-kube-api-access-28s2w\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.217836 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.217831 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.218126 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.217854 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5546c5e-1ba0-4c67-a465-8f8e612129a1-config-out\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.218126 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.217928 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.218126 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.217955 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.218126 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.217979 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.218126 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.218000 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.218126 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.218106 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.218306 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.218132 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5546c5e-1ba0-4c67-a465-8f8e612129a1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.218306 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.218156 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.218306 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.218173 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.319340 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319294 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.319575 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319340 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5546c5e-1ba0-4c67-a465-8f8e612129a1-config-out\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.319575 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319410 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.319575 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319433 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.319575 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319459 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.319575 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319487 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.319575 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319564 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.319862 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319596 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5546c5e-1ba0-4c67-a465-8f8e612129a1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.319862 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319635 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.319862 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319661 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.319862 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319698 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-config\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.319862 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319722 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.319862 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319755 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5546c5e-1ba0-4c67-a465-8f8e612129a1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.319862 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319787 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.319862 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319834 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.320230 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319863 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-web-config\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.320230 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319890 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.320230 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319920 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.320230 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319947 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.320230 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.319999 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28s2w\" (UniqueName: \"kubernetes.io/projected/f5546c5e-1ba0-4c67-a465-8f8e612129a1-kube-api-access-28s2w\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.320230 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.320200 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.320524 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.320293 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.320524 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.320410 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.320524 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.320484 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f5546c5e-1ba0-4c67-a465-8f8e612129a1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.321275 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.321026 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.321498 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.321435 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.321650 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.321622 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.323399 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.323331 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.323854 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.323804 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f5546c5e-1ba0-4c67-a465-8f8e612129a1-config-out\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.325025 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.324561 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.325025 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.324899 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f5546c5e-1ba0-4c67-a465-8f8e612129a1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.325025 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.324981 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-web-config\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.325232 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.325141 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.325575 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.325550 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.326414 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.326371 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.327490 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.327464 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-config\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.327600 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.327533 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.327810 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.327783 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f5546c5e-1ba0-4c67-a465-8f8e612129a1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.329918 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.329875 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f5546c5e-1ba0-4c67-a465-8f8e612129a1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.330428 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.330408 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28s2w\" (UniqueName: \"kubernetes.io/projected/f5546c5e-1ba0-4c67-a465-8f8e612129a1-kube-api-access-28s2w\") pod \"prometheus-k8s-0\" (UID: \"f5546c5e-1ba0-4c67-a465-8f8e612129a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.464845 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.464756 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:40.617717 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:40.617677 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 22 01:38:40.623309 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:38:40.623278 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5546c5e_1ba0_4c67_a465_8f8e612129a1.slice/crio-feb4bb12f31df6569a88dd6e5d70915784b6511825c91e5cc4c99cea2f07b07b WatchSource:0}: Error finding container feb4bb12f31df6569a88dd6e5d70915784b6511825c91e5cc4c99cea2f07b07b: Status 404 returned error can't find the container with id feb4bb12f31df6569a88dd6e5d70915784b6511825c91e5cc4c99cea2f07b07b Jul 22 01:38:41.267614 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:41.267570 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5546c5e-1ba0-4c67-a465-8f8e612129a1","Type":"ContainerStarted","Data":"feb4bb12f31df6569a88dd6e5d70915784b6511825c91e5cc4c99cea2f07b07b"} Jul 22 01:38:42.273185 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:42.273146 2559 generic.go:356] "Generic (PLEG): container finished" podID="f5546c5e-1ba0-4c67-a465-8f8e612129a1" containerID="dd1056c41894d301970e70c921c78a2e187f0209996e10f60126f6ef321b1c58" exitCode=0 Jul 22 01:38:42.273746 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:42.273237 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5546c5e-1ba0-4c67-a465-8f8e612129a1","Type":"ContainerDied","Data":"dd1056c41894d301970e70c921c78a2e187f0209996e10f60126f6ef321b1c58"} Jul 22 01:38:46.296605 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:46.296547 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5546c5e-1ba0-4c67-a465-8f8e612129a1","Type":"ContainerStarted","Data":"eda6202b0aa4312b503e47e3def634ce33b6709742a61a0556505c2c8b81a651"} Jul 22 01:38:46.297087 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:46.296647 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5546c5e-1ba0-4c67-a465-8f8e612129a1","Type":"ContainerStarted","Data":"8cae107c6efbf4d3e1b9cd86f565c552ed575d3b613caf5631a0513458727411"} Jul 22 01:38:48.308538 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:48.308501 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5546c5e-1ba0-4c67-a465-8f8e612129a1","Type":"ContainerStarted","Data":"4402f725132de8bd449587bbca56c8e312be532472937f2ba0c6bdd6c3299a3b"} Jul 22 01:38:48.309035 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:48.308577 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5546c5e-1ba0-4c67-a465-8f8e612129a1","Type":"ContainerStarted","Data":"ff49b2cf61a0e486ab609dfb47706ba3f09675202df70cc9cb3381afb418e6ae"} Jul 22 01:38:48.309035 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:48.308623 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5546c5e-1ba0-4c67-a465-8f8e612129a1","Type":"ContainerStarted","Data":"3ee11dff3a62317096be14d7392204e296ad76a78993bd0c0a42ea3847f80acc"} Jul 22 01:38:48.309035 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:48.308662 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f5546c5e-1ba0-4c67-a465-8f8e612129a1","Type":"ContainerStarted","Data":"df82733bff7de1b055e09798d7afdb4a49ff35f8413b2f7025d461e74ceba766"} Jul 22 01:38:48.343978 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:48.343910 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.264716443 podStartE2EDuration="8.343886356s" totalImagesPullingTime="6.060058206s" totalInitContainerRuntime="19.111707ms" isStatefulPod=false podCreationTimestamp="2026-07-22 01:38:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:38:48.342241399 +0000 UTC m=+113.171489746" watchObservedRunningTime="2026-07-22 01:38:48.343886356 +0000 UTC m=+113.173134705" Jul 22 01:38:49.489862 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:49.489823 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c9fc84455-szd78"] Jul 22 01:38:50.465557 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:50.465516 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:38:58.351658 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:58.351623 2559 generic.go:356] "Generic (PLEG): container finished" podID="f721d6eb-f694-4e62-9133-12edd4f6c81c" containerID="bd771db56ff5e30690d328dc384aec9f090ec745f349f1a63bf807ae8300660b" exitCode=0 Jul 22 01:38:58.352114 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:58.351690 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6jd8v" event={"ID":"f721d6eb-f694-4e62-9133-12edd4f6c81c","Type":"ContainerDied","Data":"bd771db56ff5e30690d328dc384aec9f090ec745f349f1a63bf807ae8300660b"} Jul 22 01:38:58.352114 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:58.351997 2559 scope.go:117] "RemoveContainer" containerID="bd771db56ff5e30690d328dc384aec9f090ec745f349f1a63bf807ae8300660b" Jul 22 01:38:59.357940 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:38:59.357904 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6jd8v" event={"ID":"f721d6eb-f694-4e62-9133-12edd4f6c81c","Type":"ContainerStarted","Data":"cd1f8e8f77e288d6974db67dd7ef3333887ca321b799cde0f66904d9296e7b7c"} Jul 22 01:39:01.290807 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.290739 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-687855c694-q2wpf" podUID="8ea58868-1c24-48f4-8c3d-2427d0a6dbe2" containerName="registry" containerID="cri-o://16cc5e96ac4c715b0558a9fcc017877f53fe156cfa187e1fac60788e89329f01" gracePeriod=30 Jul 22 01:39:01.548623 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.548552 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:39:01.618745 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.618713 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls\") pod \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " Jul 22 01:39:01.618745 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.618752 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-trusted-ca\") pod \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " Jul 22 01:39:01.619016 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.618772 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-bound-sa-token\") pod \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " Jul 22 01:39:01.619016 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.618795 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-certificates\") pod \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " Jul 22 01:39:01.619016 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.618844 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzx75\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-kube-api-access-xzx75\") pod \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " Jul 22 01:39:01.619016 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.618874 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-installation-pull-secrets\") pod \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " Jul 22 01:39:01.619016 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.618909 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-ca-trust-extracted\") pod \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " Jul 22 01:39:01.619016 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.618948 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-image-registry-private-configuration\") pod \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\" (UID: \"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2\") " Jul 22 01:39:01.619312 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.619240 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2" (UID: "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 01:39:01.619312 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.619290 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2" (UID: "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 01:39:01.621843 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.621808 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2" (UID: "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 01:39:01.621989 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.621858 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2" (UID: "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 01:39:01.621989 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.621900 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2" (UID: "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 01:39:01.621989 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.621947 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-kube-api-access-xzx75" (OuterVolumeSpecName: "kube-api-access-xzx75") pod "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2" (UID: "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2"). InnerVolumeSpecName "kube-api-access-xzx75". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 01:39:01.622157 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.622011 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2" (UID: "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 01:39:01.628283 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.628247 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2" (UID: "8ea58868-1c24-48f4-8c3d-2427d0a6dbe2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 01:39:01.720215 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.720171 2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-ca-trust-extracted\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:39:01.720215 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.720205 2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-image-registry-private-configuration\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:39:01.720215 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.720220 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-tls\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:39:01.720514 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.720232 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-trusted-ca\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:39:01.720514 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.720245 2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-bound-sa-token\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:39:01.720514 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.720255 2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-registry-certificates\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:39:01.720514 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.720267 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzx75\" (UniqueName: \"kubernetes.io/projected/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-kube-api-access-xzx75\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:39:01.720514 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:01.720278 2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2-installation-pull-secrets\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:39:02.371848 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:02.371810 2559 generic.go:356] "Generic (PLEG): container finished" podID="8ea58868-1c24-48f4-8c3d-2427d0a6dbe2" containerID="16cc5e96ac4c715b0558a9fcc017877f53fe156cfa187e1fac60788e89329f01" exitCode=0 Jul 22 01:39:02.371848 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:02.371852 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-687855c694-q2wpf" Jul 22 01:39:02.372312 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:02.371861 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-687855c694-q2wpf" event={"ID":"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2","Type":"ContainerDied","Data":"16cc5e96ac4c715b0558a9fcc017877f53fe156cfa187e1fac60788e89329f01"} Jul 22 01:39:02.372312 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:02.371909 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-687855c694-q2wpf" event={"ID":"8ea58868-1c24-48f4-8c3d-2427d0a6dbe2","Type":"ContainerDied","Data":"c9b117bc326f6553a153227e7b791418165ea746a3f61b6354ea1e60cb13f81f"} Jul 22 01:39:02.372312 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:02.371932 2559 scope.go:117] "RemoveContainer" containerID="16cc5e96ac4c715b0558a9fcc017877f53fe156cfa187e1fac60788e89329f01" Jul 22 01:39:02.383408 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:02.383382 2559 scope.go:117] "RemoveContainer" containerID="16cc5e96ac4c715b0558a9fcc017877f53fe156cfa187e1fac60788e89329f01" Jul 22 01:39:02.383757 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:39:02.383730 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16cc5e96ac4c715b0558a9fcc017877f53fe156cfa187e1fac60788e89329f01\": container with ID starting with 16cc5e96ac4c715b0558a9fcc017877f53fe156cfa187e1fac60788e89329f01 not found: ID does not exist" containerID="16cc5e96ac4c715b0558a9fcc017877f53fe156cfa187e1fac60788e89329f01" Jul 22 01:39:02.383858 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:02.383762 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16cc5e96ac4c715b0558a9fcc017877f53fe156cfa187e1fac60788e89329f01"} err="failed to get container status \"16cc5e96ac4c715b0558a9fcc017877f53fe156cfa187e1fac60788e89329f01\": rpc error: code = NotFound desc = could not find container \"16cc5e96ac4c715b0558a9fcc017877f53fe156cfa187e1fac60788e89329f01\": container with ID starting with 16cc5e96ac4c715b0558a9fcc017877f53fe156cfa187e1fac60788e89329f01 not found: ID does not exist" Jul 22 01:39:02.396675 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:02.396639 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-687855c694-q2wpf"] Jul 22 01:39:02.403632 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:02.403599 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-687855c694-q2wpf"] Jul 22 01:39:03.625140 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:03.625102 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ea58868-1c24-48f4-8c3d-2427d0a6dbe2" path="/var/lib/kubelet/pods/8ea58868-1c24-48f4-8c3d-2427d0a6dbe2/volumes" Jul 22 01:39:05.452932 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:05.452882 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs\") pod \"network-metrics-daemon-878hc\" (UID: \"8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e\") " pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:39:05.455304 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:05.455275 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e-metrics-certs\") pod \"network-metrics-daemon-878hc\" (UID: \"8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e\") " pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:39:05.539189 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:05.539155 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fbrqb\"" Jul 22 01:39:05.547653 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:05.547626 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-878hc" Jul 22 01:39:05.879618 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:05.879581 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-878hc"] Jul 22 01:39:05.883057 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:39:05.883013 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8de9ff3f_ccb1_4eb5_9dbf_4c29109fae6e.slice/crio-c9784138febf78fc06ae961f62551d596660a9e5477bedf47df817e2241c4a12 WatchSource:0}: Error finding container c9784138febf78fc06ae961f62551d596660a9e5477bedf47df817e2241c4a12: Status 404 returned error can't find the container with id c9784138febf78fc06ae961f62551d596660a9e5477bedf47df817e2241c4a12 Jul 22 01:39:06.391449 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:06.391402 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-878hc" event={"ID":"8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e","Type":"ContainerStarted","Data":"c9784138febf78fc06ae961f62551d596660a9e5477bedf47df817e2241c4a12"} Jul 22 01:39:08.401321 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:08.401284 2559 generic.go:356] "Generic (PLEG): container finished" podID="6b7342d6-764b-45cf-b848-9e5a219b073c" containerID="3b78f846c7ce212d8ce410fede754d785e172ad488930027e8c5095d8d6f5cf9" exitCode=0 Jul 22 01:39:08.401772 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:08.401379 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wjp2" event={"ID":"6b7342d6-764b-45cf-b848-9e5a219b073c","Type":"ContainerDied","Data":"3b78f846c7ce212d8ce410fede754d785e172ad488930027e8c5095d8d6f5cf9"} Jul 22 01:39:08.401832 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:08.401773 2559 scope.go:117] "RemoveContainer" containerID="3b78f846c7ce212d8ce410fede754d785e172ad488930027e8c5095d8d6f5cf9" Jul 22 01:39:08.403328 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:08.403290 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-878hc" event={"ID":"8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e","Type":"ContainerStarted","Data":"edb80514996a284cb933b8f904f0b7892338817074047e08e2b681b3981767d0"} Jul 22 01:39:08.403478 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:08.403400 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-878hc" event={"ID":"8de9ff3f-ccb1-4eb5-9dbf-4c29109fae6e","Type":"ContainerStarted","Data":"24de98cd986e3fd30d3812d5ca358a647944cefaa282d9a2a5ea559c4ae9244e"} Jul 22 01:39:08.404893 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:08.404822 2559 generic.go:356] "Generic (PLEG): container finished" podID="2f7b6603-7aa5-411a-8956-40e0982d98f4" containerID="2751b2a5511bb1983de4623727c3845a9d278b2a7a68ee937cf9b88256a3d5e9" exitCode=0 Jul 22 01:39:08.404992 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:08.404919 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-flhhx" event={"ID":"2f7b6603-7aa5-411a-8956-40e0982d98f4","Type":"ContainerDied","Data":"2751b2a5511bb1983de4623727c3845a9d278b2a7a68ee937cf9b88256a3d5e9"} Jul 22 01:39:08.405218 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:08.405204 2559 scope.go:117] "RemoveContainer" containerID="2751b2a5511bb1983de4623727c3845a9d278b2a7a68ee937cf9b88256a3d5e9" Jul 22 01:39:08.438816 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:08.438718 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-878hc" podStartSLOduration=131.790718258 podStartE2EDuration="2m13.4386984s" totalImagesPullingTime="1.647980142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:36:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:39:08.438037723 +0000 UTC m=+133.267286073" watchObservedRunningTime="2026-07-22 01:39:08.4386984 +0000 UTC m=+133.267946747" Jul 22 01:39:08.971251 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:08.971220 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-fb4497b6-psq7s_e694641c-026f-4a5f-918b-6fedb0b96cdd/router/0.log" Jul 22 01:39:09.000001 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:08.999966 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qhr55_0df0a233-5828-4368-96ca-134ab7641aed/serve-healthcheck-canary/0.log" Jul 22 01:39:09.413232 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:09.413193 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5wjp2" event={"ID":"6b7342d6-764b-45cf-b848-9e5a219b073c","Type":"ContainerStarted","Data":"c77592fef9ace81f9b2c320758f57c973103f5be4d79853427f84f1853e5b04d"} Jul 22 01:39:09.414985 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:09.414953 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-flhhx" event={"ID":"2f7b6603-7aa5-411a-8956-40e0982d98f4","Type":"ContainerStarted","Data":"904443b4dc09e73f26b43ef4259a4065707fe76c3c9515066ee8076f5b228cbe"} Jul 22 01:39:14.513072 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.513026 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c9fc84455-szd78" podUID="346ef971-b332-4cd5-801c-28a47c1fa213" containerName="console" containerID="cri-o://a3b87065d9e733574322dbb26e6f7acd75713dd1993878d9bf63be4fb9cfc0ff" gracePeriod=15 Jul 22 01:39:14.765734 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.765664 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:39:14.842578 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.842527 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lmmb\" (UniqueName: \"kubernetes.io/projected/346ef971-b332-4cd5-801c-28a47c1fa213-kube-api-access-6lmmb\") pod \"346ef971-b332-4cd5-801c-28a47c1fa213\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " Jul 22 01:39:14.842578 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.842592 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/346ef971-b332-4cd5-801c-28a47c1fa213-console-oauth-config\") pod \"346ef971-b332-4cd5-801c-28a47c1fa213\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " Jul 22 01:39:14.842804 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.842642 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-console-config\") pod \"346ef971-b332-4cd5-801c-28a47c1fa213\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " Jul 22 01:39:14.842804 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.842681 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-service-ca\") pod \"346ef971-b332-4cd5-801c-28a47c1fa213\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " Jul 22 01:39:14.842867 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.842807 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/346ef971-b332-4cd5-801c-28a47c1fa213-console-serving-cert\") pod \"346ef971-b332-4cd5-801c-28a47c1fa213\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " Jul 22 01:39:14.842867 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.842848 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-trusted-ca-bundle\") pod \"346ef971-b332-4cd5-801c-28a47c1fa213\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " Jul 22 01:39:14.842965 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.842884 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-oauth-serving-cert\") pod \"346ef971-b332-4cd5-801c-28a47c1fa213\" (UID: \"346ef971-b332-4cd5-801c-28a47c1fa213\") " Jul 22 01:39:14.843067 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.843024 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-console-config" (OuterVolumeSpecName: "console-config") pod "346ef971-b332-4cd5-801c-28a47c1fa213" (UID: "346ef971-b332-4cd5-801c-28a47c1fa213"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 01:39:14.843173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.843044 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-service-ca" (OuterVolumeSpecName: "service-ca") pod "346ef971-b332-4cd5-801c-28a47c1fa213" (UID: "346ef971-b332-4cd5-801c-28a47c1fa213"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 01:39:14.843230 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.843202 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-console-config\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:39:14.843281 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.843243 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "346ef971-b332-4cd5-801c-28a47c1fa213" (UID: "346ef971-b332-4cd5-801c-28a47c1fa213"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 01:39:14.843390 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.843367 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "346ef971-b332-4cd5-801c-28a47c1fa213" (UID: "346ef971-b332-4cd5-801c-28a47c1fa213"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 01:39:14.845025 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.845004 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/346ef971-b332-4cd5-801c-28a47c1fa213-kube-api-access-6lmmb" (OuterVolumeSpecName: "kube-api-access-6lmmb") pod "346ef971-b332-4cd5-801c-28a47c1fa213" (UID: "346ef971-b332-4cd5-801c-28a47c1fa213"). InnerVolumeSpecName "kube-api-access-6lmmb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 01:39:14.845268 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.845248 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/346ef971-b332-4cd5-801c-28a47c1fa213-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "346ef971-b332-4cd5-801c-28a47c1fa213" (UID: "346ef971-b332-4cd5-801c-28a47c1fa213"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 01:39:14.845315 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.845269 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/346ef971-b332-4cd5-801c-28a47c1fa213-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "346ef971-b332-4cd5-801c-28a47c1fa213" (UID: "346ef971-b332-4cd5-801c-28a47c1fa213"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 01:39:14.944053 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.944002 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/346ef971-b332-4cd5-801c-28a47c1fa213-console-oauth-config\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:39:14.944053 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.944048 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-service-ca\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:39:14.944053 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.944059 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/346ef971-b332-4cd5-801c-28a47c1fa213-console-serving-cert\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:39:14.944053 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.944071 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-trusted-ca-bundle\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:39:14.944336 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.944080 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/346ef971-b332-4cd5-801c-28a47c1fa213-oauth-serving-cert\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:39:14.944336 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:14.944090 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lmmb\" (UniqueName: \"kubernetes.io/projected/346ef971-b332-4cd5-801c-28a47c1fa213-kube-api-access-6lmmb\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:39:15.441533 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:15.441495 2559 generic.go:356] "Generic (PLEG): container finished" podID="346ef971-b332-4cd5-801c-28a47c1fa213" containerID="a3b87065d9e733574322dbb26e6f7acd75713dd1993878d9bf63be4fb9cfc0ff" exitCode=0 Jul 22 01:39:15.441744 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:15.441542 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c9fc84455-szd78" Jul 22 01:39:15.441744 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:15.441566 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c9fc84455-szd78" event={"ID":"346ef971-b332-4cd5-801c-28a47c1fa213","Type":"ContainerDied","Data":"a3b87065d9e733574322dbb26e6f7acd75713dd1993878d9bf63be4fb9cfc0ff"} Jul 22 01:39:15.441744 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:15.441616 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c9fc84455-szd78" event={"ID":"346ef971-b332-4cd5-801c-28a47c1fa213","Type":"ContainerDied","Data":"ed690fff4df4606cafed49a0d22f5c616d98b535259ba706a13ec7a8a3288cfa"} Jul 22 01:39:15.441744 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:15.441641 2559 scope.go:117] "RemoveContainer" containerID="a3b87065d9e733574322dbb26e6f7acd75713dd1993878d9bf63be4fb9cfc0ff" Jul 22 01:39:15.451160 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:15.451141 2559 scope.go:117] "RemoveContainer" containerID="a3b87065d9e733574322dbb26e6f7acd75713dd1993878d9bf63be4fb9cfc0ff" Jul 22 01:39:15.451510 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:39:15.451488 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3b87065d9e733574322dbb26e6f7acd75713dd1993878d9bf63be4fb9cfc0ff\": container with ID starting with a3b87065d9e733574322dbb26e6f7acd75713dd1993878d9bf63be4fb9cfc0ff not found: ID does not exist" containerID="a3b87065d9e733574322dbb26e6f7acd75713dd1993878d9bf63be4fb9cfc0ff" Jul 22 01:39:15.451587 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:15.451518 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3b87065d9e733574322dbb26e6f7acd75713dd1993878d9bf63be4fb9cfc0ff"} err="failed to get container status \"a3b87065d9e733574322dbb26e6f7acd75713dd1993878d9bf63be4fb9cfc0ff\": rpc error: code = NotFound desc = could not find container \"a3b87065d9e733574322dbb26e6f7acd75713dd1993878d9bf63be4fb9cfc0ff\": container with ID starting with a3b87065d9e733574322dbb26e6f7acd75713dd1993878d9bf63be4fb9cfc0ff not found: ID does not exist" Jul 22 01:39:15.468068 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:15.468033 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c9fc84455-szd78"] Jul 22 01:39:15.472297 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:15.472268 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c9fc84455-szd78"] Jul 22 01:39:15.625320 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:15.625281 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="346ef971-b332-4cd5-801c-28a47c1fa213" path="/var/lib/kubelet/pods/346ef971-b332-4cd5-801c-28a47c1fa213/volumes" Jul 22 01:39:40.465648 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:40.465579 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:39:40.482464 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:40.482433 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:39:40.562242 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:39:40.562210 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 01:41:55.485167 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:41:55.485141 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 01:41:55.488102 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:41:55.488082 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 01:41:55.489771 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:41:55.489752 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 01:41:55.492986 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:41:55.492968 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 01:41:55.497296 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:41:55.497273 2559 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 01:42:02.176303 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.176271 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gxnhh"] Jul 22 01:42:02.178515 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.176608 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ea58868-1c24-48f4-8c3d-2427d0a6dbe2" containerName="registry" Jul 22 01:42:02.178515 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.176620 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ea58868-1c24-48f4-8c3d-2427d0a6dbe2" containerName="registry" Jul 22 01:42:02.178515 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.176638 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="346ef971-b332-4cd5-801c-28a47c1fa213" containerName="console" Jul 22 01:42:02.178515 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.176644 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="346ef971-b332-4cd5-801c-28a47c1fa213" containerName="console" Jul 22 01:42:02.178515 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.176692 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ea58868-1c24-48f4-8c3d-2427d0a6dbe2" containerName="registry" Jul 22 01:42:02.178515 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.176700 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="346ef971-b332-4cd5-801c-28a47c1fa213" containerName="console" Jul 22 01:42:02.179387 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.179371 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gxnhh" Jul 22 01:42:02.182416 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.182396 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 22 01:42:02.182950 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.182922 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-5sdpf\"" Jul 22 01:42:02.182950 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.182941 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 22 01:42:02.193621 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.193597 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gxnhh"] Jul 22 01:42:02.207837 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.207815 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e09d8012-31ad-42cc-a89d-7eb482723777-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gxnhh\" (UID: \"e09d8012-31ad-42cc-a89d-7eb482723777\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gxnhh" Jul 22 01:42:02.207935 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.207864 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krql7\" (UniqueName: \"kubernetes.io/projected/e09d8012-31ad-42cc-a89d-7eb482723777-kube-api-access-krql7\") pod \"openshift-lws-operator-fd8ccff4c-gxnhh\" (UID: \"e09d8012-31ad-42cc-a89d-7eb482723777\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gxnhh" Jul 22 01:42:02.308553 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.308523 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krql7\" (UniqueName: \"kubernetes.io/projected/e09d8012-31ad-42cc-a89d-7eb482723777-kube-api-access-krql7\") pod \"openshift-lws-operator-fd8ccff4c-gxnhh\" (UID: \"e09d8012-31ad-42cc-a89d-7eb482723777\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gxnhh" Jul 22 01:42:02.308674 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.308579 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e09d8012-31ad-42cc-a89d-7eb482723777-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gxnhh\" (UID: \"e09d8012-31ad-42cc-a89d-7eb482723777\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gxnhh" Jul 22 01:42:02.308894 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.308878 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e09d8012-31ad-42cc-a89d-7eb482723777-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gxnhh\" (UID: \"e09d8012-31ad-42cc-a89d-7eb482723777\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gxnhh" Jul 22 01:42:02.317272 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.317250 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krql7\" (UniqueName: \"kubernetes.io/projected/e09d8012-31ad-42cc-a89d-7eb482723777-kube-api-access-krql7\") pod \"openshift-lws-operator-fd8ccff4c-gxnhh\" (UID: \"e09d8012-31ad-42cc-a89d-7eb482723777\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gxnhh" Jul 22 01:42:02.497210 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.497125 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gxnhh" Jul 22 01:42:02.629476 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.629446 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gxnhh"] Jul 22 01:42:02.632642 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:42:02.632615 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode09d8012_31ad_42cc_a89d_7eb482723777.slice/crio-e4289e2c0117b15accb5535cf8a9592304666e69da64817b0706775cab4fed94 WatchSource:0}: Error finding container e4289e2c0117b15accb5535cf8a9592304666e69da64817b0706775cab4fed94: Status 404 returned error can't find the container with id e4289e2c0117b15accb5535cf8a9592304666e69da64817b0706775cab4fed94 Jul 22 01:42:02.634222 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:02.634197 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 01:42:03.122915 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:03.122880 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gxnhh" event={"ID":"e09d8012-31ad-42cc-a89d-7eb482723777","Type":"ContainerStarted","Data":"e4289e2c0117b15accb5535cf8a9592304666e69da64817b0706775cab4fed94"} Jul 22 01:42:07.140502 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:07.140465 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gxnhh" event={"ID":"e09d8012-31ad-42cc-a89d-7eb482723777","Type":"ContainerStarted","Data":"dfe798e4e019ed2bf2f71a85798875d77cc766797cf4c9ad321772d75e6083be"} Jul 22 01:42:07.157617 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:07.157566 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gxnhh" podStartSLOduration=1.631621697 podStartE2EDuration="5.15755208s" totalImagesPullingTime="3.525930383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:42:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:42:07.155919898 +0000 UTC m=+311.985168247" watchObservedRunningTime="2026-07-22 01:42:07.15755208 +0000 UTC m=+311.986800427" Jul 22 01:42:24.163138 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.163102 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-65486778df-42clz"] Jul 22 01:42:24.170166 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.170141 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65486778df-42clz" Jul 22 01:42:24.174213 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.174188 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 22 01:42:24.174213 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.174202 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 22 01:42:24.174406 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.174212 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 22 01:42:24.174406 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.174230 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jqptx\"" Jul 22 01:42:24.184516 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.184488 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65486778df-42clz"] Jul 22 01:42:24.292523 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.292484 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f584425a-5e67-4b6b-82bb-67bfc7a8bdad-cert\") pod \"lws-controller-manager-65486778df-42clz\" (UID: \"f584425a-5e67-4b6b-82bb-67bfc7a8bdad\") " pod="openshift-lws-operator/lws-controller-manager-65486778df-42clz" Jul 22 01:42:24.292523 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.292527 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f584425a-5e67-4b6b-82bb-67bfc7a8bdad-metrics-cert\") pod \"lws-controller-manager-65486778df-42clz\" (UID: \"f584425a-5e67-4b6b-82bb-67bfc7a8bdad\") " pod="openshift-lws-operator/lws-controller-manager-65486778df-42clz" Jul 22 01:42:24.292719 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.292552 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f584425a-5e67-4b6b-82bb-67bfc7a8bdad-manager-config\") pod \"lws-controller-manager-65486778df-42clz\" (UID: \"f584425a-5e67-4b6b-82bb-67bfc7a8bdad\") " pod="openshift-lws-operator/lws-controller-manager-65486778df-42clz" Jul 22 01:42:24.292719 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.292630 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smhc2\" (UniqueName: \"kubernetes.io/projected/f584425a-5e67-4b6b-82bb-67bfc7a8bdad-kube-api-access-smhc2\") pod \"lws-controller-manager-65486778df-42clz\" (UID: \"f584425a-5e67-4b6b-82bb-67bfc7a8bdad\") " pod="openshift-lws-operator/lws-controller-manager-65486778df-42clz" Jul 22 01:42:24.393226 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.393185 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f584425a-5e67-4b6b-82bb-67bfc7a8bdad-cert\") pod \"lws-controller-manager-65486778df-42clz\" (UID: \"f584425a-5e67-4b6b-82bb-67bfc7a8bdad\") " pod="openshift-lws-operator/lws-controller-manager-65486778df-42clz" Jul 22 01:42:24.393457 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.393233 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f584425a-5e67-4b6b-82bb-67bfc7a8bdad-metrics-cert\") pod \"lws-controller-manager-65486778df-42clz\" (UID: \"f584425a-5e67-4b6b-82bb-67bfc7a8bdad\") " pod="openshift-lws-operator/lws-controller-manager-65486778df-42clz" Jul 22 01:42:24.393457 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.393269 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f584425a-5e67-4b6b-82bb-67bfc7a8bdad-manager-config\") pod \"lws-controller-manager-65486778df-42clz\" (UID: \"f584425a-5e67-4b6b-82bb-67bfc7a8bdad\") " pod="openshift-lws-operator/lws-controller-manager-65486778df-42clz" Jul 22 01:42:24.393457 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.393299 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smhc2\" (UniqueName: \"kubernetes.io/projected/f584425a-5e67-4b6b-82bb-67bfc7a8bdad-kube-api-access-smhc2\") pod \"lws-controller-manager-65486778df-42clz\" (UID: \"f584425a-5e67-4b6b-82bb-67bfc7a8bdad\") " pod="openshift-lws-operator/lws-controller-manager-65486778df-42clz" Jul 22 01:42:24.394095 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.394073 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f584425a-5e67-4b6b-82bb-67bfc7a8bdad-manager-config\") pod \"lws-controller-manager-65486778df-42clz\" (UID: \"f584425a-5e67-4b6b-82bb-67bfc7a8bdad\") " pod="openshift-lws-operator/lws-controller-manager-65486778df-42clz" Jul 22 01:42:24.395673 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.395654 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f584425a-5e67-4b6b-82bb-67bfc7a8bdad-cert\") pod \"lws-controller-manager-65486778df-42clz\" (UID: \"f584425a-5e67-4b6b-82bb-67bfc7a8bdad\") " pod="openshift-lws-operator/lws-controller-manager-65486778df-42clz" Jul 22 01:42:24.395738 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.395699 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f584425a-5e67-4b6b-82bb-67bfc7a8bdad-metrics-cert\") pod \"lws-controller-manager-65486778df-42clz\" (UID: \"f584425a-5e67-4b6b-82bb-67bfc7a8bdad\") " pod="openshift-lws-operator/lws-controller-manager-65486778df-42clz" Jul 22 01:42:24.404476 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.404458 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smhc2\" (UniqueName: \"kubernetes.io/projected/f584425a-5e67-4b6b-82bb-67bfc7a8bdad-kube-api-access-smhc2\") pod \"lws-controller-manager-65486778df-42clz\" (UID: \"f584425a-5e67-4b6b-82bb-67bfc7a8bdad\") " pod="openshift-lws-operator/lws-controller-manager-65486778df-42clz" Jul 22 01:42:24.479719 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.479641 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65486778df-42clz" Jul 22 01:42:24.608933 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:24.608909 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65486778df-42clz"] Jul 22 01:42:24.611550 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:42:24.611517 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf584425a_5e67_4b6b_82bb_67bfc7a8bdad.slice/crio-d25210e1c1cf357047e88028d674c01197718ae1274e9eb1c528c7d5a13d075b WatchSource:0}: Error finding container d25210e1c1cf357047e88028d674c01197718ae1274e9eb1c528c7d5a13d075b: Status 404 returned error can't find the container with id d25210e1c1cf357047e88028d674c01197718ae1274e9eb1c528c7d5a13d075b Jul 22 01:42:25.218155 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:25.218115 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65486778df-42clz" event={"ID":"f584425a-5e67-4b6b-82bb-67bfc7a8bdad","Type":"ContainerStarted","Data":"d25210e1c1cf357047e88028d674c01197718ae1274e9eb1c528c7d5a13d075b"} Jul 22 01:42:26.532070 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.532033 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b"] Jul 22 01:42:26.535895 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.535873 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" Jul 22 01:42:26.538333 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.538302 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 22 01:42:26.538460 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.538404 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 22 01:42:26.538460 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.538404 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 22 01:42:26.538581 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.538521 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6cfrp\"" Jul 22 01:42:26.538581 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.538532 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 22 01:42:26.550597 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.550574 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b"] Jul 22 01:42:26.614496 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.614462 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07b210d8-f291-4212-83c5-b2e6398a55b5-tmp\") pod \"opendatahub-operator-controller-manager-5c578697c6-xf47b\" (UID: \"07b210d8-f291-4212-83c5-b2e6398a55b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" Jul 22 01:42:26.614496 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.614499 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/07b210d8-f291-4212-83c5-b2e6398a55b5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c578697c6-xf47b\" (UID: \"07b210d8-f291-4212-83c5-b2e6398a55b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" Jul 22 01:42:26.614989 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.614563 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44tmd\" (UniqueName: \"kubernetes.io/projected/07b210d8-f291-4212-83c5-b2e6398a55b5-kube-api-access-44tmd\") pod \"opendatahub-operator-controller-manager-5c578697c6-xf47b\" (UID: \"07b210d8-f291-4212-83c5-b2e6398a55b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" Jul 22 01:42:26.614989 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.614613 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/07b210d8-f291-4212-83c5-b2e6398a55b5-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c578697c6-xf47b\" (UID: \"07b210d8-f291-4212-83c5-b2e6398a55b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" Jul 22 01:42:26.614989 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.614634 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/07b210d8-f291-4212-83c5-b2e6398a55b5-manifests\") pod \"opendatahub-operator-controller-manager-5c578697c6-xf47b\" (UID: \"07b210d8-f291-4212-83c5-b2e6398a55b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" Jul 22 01:42:26.715644 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.715605 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07b210d8-f291-4212-83c5-b2e6398a55b5-tmp\") pod \"opendatahub-operator-controller-manager-5c578697c6-xf47b\" (UID: \"07b210d8-f291-4212-83c5-b2e6398a55b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" Jul 22 01:42:26.715844 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.715655 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/07b210d8-f291-4212-83c5-b2e6398a55b5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c578697c6-xf47b\" (UID: \"07b210d8-f291-4212-83c5-b2e6398a55b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" Jul 22 01:42:26.715844 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.715690 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44tmd\" (UniqueName: \"kubernetes.io/projected/07b210d8-f291-4212-83c5-b2e6398a55b5-kube-api-access-44tmd\") pod \"opendatahub-operator-controller-manager-5c578697c6-xf47b\" (UID: \"07b210d8-f291-4212-83c5-b2e6398a55b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" Jul 22 01:42:26.715844 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.715718 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/07b210d8-f291-4212-83c5-b2e6398a55b5-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c578697c6-xf47b\" (UID: \"07b210d8-f291-4212-83c5-b2e6398a55b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" Jul 22 01:42:26.715844 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.715741 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/07b210d8-f291-4212-83c5-b2e6398a55b5-manifests\") pod \"opendatahub-operator-controller-manager-5c578697c6-xf47b\" (UID: \"07b210d8-f291-4212-83c5-b2e6398a55b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" Jul 22 01:42:26.716136 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.716115 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/07b210d8-f291-4212-83c5-b2e6398a55b5-manifests\") pod \"opendatahub-operator-controller-manager-5c578697c6-xf47b\" (UID: \"07b210d8-f291-4212-83c5-b2e6398a55b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" Jul 22 01:42:26.718290 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.718261 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/07b210d8-f291-4212-83c5-b2e6398a55b5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c578697c6-xf47b\" (UID: \"07b210d8-f291-4212-83c5-b2e6398a55b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" Jul 22 01:42:26.718503 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.718481 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07b210d8-f291-4212-83c5-b2e6398a55b5-tmp\") pod \"opendatahub-operator-controller-manager-5c578697c6-xf47b\" (UID: \"07b210d8-f291-4212-83c5-b2e6398a55b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" Jul 22 01:42:26.718588 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.718568 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/07b210d8-f291-4212-83c5-b2e6398a55b5-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c578697c6-xf47b\" (UID: \"07b210d8-f291-4212-83c5-b2e6398a55b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" Jul 22 01:42:26.723791 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.723770 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44tmd\" (UniqueName: \"kubernetes.io/projected/07b210d8-f291-4212-83c5-b2e6398a55b5-kube-api-access-44tmd\") pod \"opendatahub-operator-controller-manager-5c578697c6-xf47b\" (UID: \"07b210d8-f291-4212-83c5-b2e6398a55b5\") " pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" Jul 22 01:42:26.847137 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:26.847101 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" Jul 22 01:42:27.157412 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:27.157377 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b"] Jul 22 01:42:27.161595 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:42:27.161552 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b210d8_f291_4212_83c5_b2e6398a55b5.slice/crio-6e599d7c5ac9a93d8070df33ac53608687c30e563ed36d188b8ab0054499128d WatchSource:0}: Error finding container 6e599d7c5ac9a93d8070df33ac53608687c30e563ed36d188b8ab0054499128d: Status 404 returned error can't find the container with id 6e599d7c5ac9a93d8070df33ac53608687c30e563ed36d188b8ab0054499128d Jul 22 01:42:27.229071 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:27.229032 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65486778df-42clz" event={"ID":"f584425a-5e67-4b6b-82bb-67bfc7a8bdad","Type":"ContainerStarted","Data":"de42160d5630ff37addcd7c578a69b1253d553552f5c6ee1130f78c0d5082c3d"} Jul 22 01:42:27.229269 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:27.229169 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-65486778df-42clz" Jul 22 01:42:27.230470 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:27.230445 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" event={"ID":"07b210d8-f291-4212-83c5-b2e6398a55b5","Type":"ContainerStarted","Data":"6e599d7c5ac9a93d8070df33ac53608687c30e563ed36d188b8ab0054499128d"} Jul 22 01:42:27.255866 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:27.255812 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-65486778df-42clz" podStartSLOduration=0.791441261 podStartE2EDuration="3.255792168s" totalImagesPullingTime="2.464350907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:42:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:42:27.254550502 +0000 UTC m=+332.083798850" watchObservedRunningTime="2026-07-22 01:42:27.255792168 +0000 UTC m=+332.085040517" Jul 22 01:42:31.251361 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:31.251320 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" event={"ID":"07b210d8-f291-4212-83c5-b2e6398a55b5","Type":"ContainerStarted","Data":"d9a6790777760d8098dd1b3b82aed2dd296c16861c4ef6e9e1a28ed8173a2483"} Jul 22 01:42:32.256561 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:32.256526 2559 generic.go:356] "Generic (PLEG): container finished" podID="07b210d8-f291-4212-83c5-b2e6398a55b5" containerID="d9a6790777760d8098dd1b3b82aed2dd296c16861c4ef6e9e1a28ed8173a2483" exitCode=0 Jul 22 01:42:32.256987 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:32.256610 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" event={"ID":"07b210d8-f291-4212-83c5-b2e6398a55b5","Type":"ContainerDied","Data":"d9a6790777760d8098dd1b3b82aed2dd296c16861c4ef6e9e1a28ed8173a2483"} Jul 22 01:42:33.263275 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:33.263240 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" event={"ID":"07b210d8-f291-4212-83c5-b2e6398a55b5","Type":"ContainerStarted","Data":"a0b4a037484b4ab0361b31fbd4d8626346c95dc643afeaeaf27cc8c33fcf29f2"} Jul 22 01:42:33.263762 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:33.263442 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" Jul 22 01:42:33.286939 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:33.286889 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" podStartSLOduration=2.658228113 podStartE2EDuration="7.286871404s" totalImagesPullingTime="3.623507715s" totalInitContainerRuntime="1.005135576s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:42:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:42:33.285120983 +0000 UTC m=+338.114369330" watchObservedRunningTime="2026-07-22 01:42:33.286871404 +0000 UTC m=+338.116119756" Jul 22 01:42:38.239014 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:38.238981 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-65486778df-42clz" Jul 22 01:42:44.270650 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:44.270619 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5c578697c6-xf47b" Jul 22 01:42:53.965147 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:53.965072 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg"] Jul 22 01:42:53.968111 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:53.968085 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:53.970328 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:53.970308 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 01:42:53.970445 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:53.970308 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-8cvms\"" Jul 22 01:42:53.981049 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:53.981022 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg"] Jul 22 01:42:54.044212 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.044173 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh"] Jul 22 01:42:54.047758 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.047739 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.056607 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.056583 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39722719-282b-48fe-be81-2472e3eaaada-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.056764 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.056614 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.056764 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.056631 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.056764 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.056652 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.056764 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.056699 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nltlg\" (UniqueName: \"kubernetes.io/projected/39722719-282b-48fe-be81-2472e3eaaada-kube-api-access-nltlg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.056764 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.056755 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.057035 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.056772 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39722719-282b-48fe-be81-2472e3eaaada-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.057035 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.056798 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.057035 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.056823 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39722719-282b-48fe-be81-2472e3eaaada-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.057035 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.056854 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.057035 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.056880 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.057035 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.056902 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39722719-282b-48fe-be81-2472e3eaaada-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.057035 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.056937 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39722719-282b-48fe-be81-2472e3eaaada-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.057035 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.056969 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fctzc\" (UniqueName: \"kubernetes.io/projected/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-kube-api-access-fctzc\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.057035 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.056988 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39722719-282b-48fe-be81-2472e3eaaada-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.057035 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.057006 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39722719-282b-48fe-be81-2472e3eaaada-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.057035 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.057037 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.057577 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.057106 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39722719-282b-48fe-be81-2472e3eaaada-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.059514 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.059494 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh"] Jul 22 01:42:54.157883 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.157847 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39722719-282b-48fe-be81-2472e3eaaada-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.157883 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.157885 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.158131 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.157904 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.158131 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.157928 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.158131 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.157957 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nltlg\" (UniqueName: \"kubernetes.io/projected/39722719-282b-48fe-be81-2472e3eaaada-kube-api-access-nltlg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.158131 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.158001 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.158131 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.158024 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39722719-282b-48fe-be81-2472e3eaaada-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.158131 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.158060 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.158466 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.158248 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39722719-282b-48fe-be81-2472e3eaaada-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.158466 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.158276 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39722719-282b-48fe-be81-2472e3eaaada-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.158466 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.158306 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.158466 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.158364 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.158466 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.158392 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39722719-282b-48fe-be81-2472e3eaaada-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.158466 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.158415 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.158466 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.158420 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39722719-282b-48fe-be81-2472e3eaaada-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.158808 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.158490 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fctzc\" (UniqueName: \"kubernetes.io/projected/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-kube-api-access-fctzc\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.158808 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.158518 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39722719-282b-48fe-be81-2472e3eaaada-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.158808 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.158542 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39722719-282b-48fe-be81-2472e3eaaada-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.158808 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.158568 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.158808 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.158589 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39722719-282b-48fe-be81-2472e3eaaada-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.158808 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.158600 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39722719-282b-48fe-be81-2472e3eaaada-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.158808 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.158670 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39722719-282b-48fe-be81-2472e3eaaada-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.158808 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.158737 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.160831 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.160789 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.161027 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.160882 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.161100 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.161019 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39722719-282b-48fe-be81-2472e3eaaada-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.161675 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.161400 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39722719-282b-48fe-be81-2472e3eaaada-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.161675 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.161406 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.161675 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.161557 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39722719-282b-48fe-be81-2472e3eaaada-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.161911 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.161707 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.161911 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.161777 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.165333 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.163799 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39722719-282b-48fe-be81-2472e3eaaada-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.167845 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.167821 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fctzc\" (UniqueName: \"kubernetes.io/projected/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-kube-api-access-fctzc\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.169464 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.169440 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39722719-282b-48fe-be81-2472e3eaaada-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.169553 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.169534 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nltlg\" (UniqueName: \"kubernetes.io/projected/39722719-282b-48fe-be81-2472e3eaaada-kube-api-access-nltlg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sljsh\" (UID: \"39722719-282b-48fe-be81-2472e3eaaada\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.170397 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.170377 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-76xlg\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.279424 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.279304 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:54.358432 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.358400 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:54.423277 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.423252 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg"] Jul 22 01:42:54.425856 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:42:54.425824 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cfb6b91_6e4b_460d_a7b3_30134fb115ce.slice/crio-d7455aa07ce9801a167c62711023c8cd640ec17e4193b7c5de42afbf387a6bc2 WatchSource:0}: Error finding container d7455aa07ce9801a167c62711023c8cd640ec17e4193b7c5de42afbf387a6bc2: Status 404 returned error can't find the container with id d7455aa07ce9801a167c62711023c8cd640ec17e4193b7c5de42afbf387a6bc2 Jul 22 01:42:54.493996 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:54.493972 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh"] Jul 22 01:42:54.496336 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:42:54.496305 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39722719_282b_48fe_be81_2472e3eaaada.slice/crio-a3c6e32a88a587bd954db6462917f401553a1a4c950c77f4099c34aa2b2f4600 WatchSource:0}: Error finding container a3c6e32a88a587bd954db6462917f401553a1a4c950c77f4099c34aa2b2f4600: Status 404 returned error can't find the container with id a3c6e32a88a587bd954db6462917f401553a1a4c950c77f4099c34aa2b2f4600 Jul 22 01:42:55.360916 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:55.360812 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" event={"ID":"7cfb6b91-6e4b-460d-a7b3-30134fb115ce","Type":"ContainerStarted","Data":"d7455aa07ce9801a167c62711023c8cd640ec17e4193b7c5de42afbf387a6bc2"} Jul 22 01:42:55.362631 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:55.362541 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" event={"ID":"39722719-282b-48fe-be81-2472e3eaaada","Type":"ContainerStarted","Data":"a3c6e32a88a587bd954db6462917f401553a1a4c950c77f4099c34aa2b2f4600"} Jul 22 01:42:56.505390 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.505338 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t"] Jul 22 01:42:56.509308 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.509282 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.511976 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.511953 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 22 01:42:56.512090 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.511953 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 22 01:42:56.512090 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.511982 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 22 01:42:56.512201 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.512109 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-24kgn\"" Jul 22 01:42:56.522661 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.522637 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t"] Jul 22 01:42:56.584067 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.584028 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.584219 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.584073 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.584219 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.584153 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.584219 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.584177 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.584219 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.584202 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.584428 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.584222 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbmtq\" (UniqueName: \"kubernetes.io/projected/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-kube-api-access-nbmtq\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.584428 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.584318 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.685626 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.685593 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.685792 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.685651 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.685792 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.685689 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.685792 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.685749 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.685792 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.685772 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.685984 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.685825 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.685984 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.685872 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbmtq\" (UniqueName: \"kubernetes.io/projected/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-kube-api-access-nbmtq\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.686519 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.686467 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.688656 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.688632 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.688764 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.688635 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.688764 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.688681 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.688764 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.688718 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.696977 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.696952 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbmtq\" (UniqueName: \"kubernetes.io/projected/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-kube-api-access-nbmtq\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.697494 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.697469 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d6c76ff-846e-41aa-b84d-b1003ca82b6b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5cw2t\" (UID: \"8d6c76ff-846e-41aa-b84d-b1003ca82b6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:56.821564 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:56.821536 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:42:57.580311 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:57.580255 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 01:42:57.580740 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:57.580373 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 01:42:57.580740 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:57.580441 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 01:42:57.589027 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:57.588907 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 01:42:57.589027 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:57.588995 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 01:42:57.589196 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:57.589041 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 01:42:57.644727 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:57.644702 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t"] Jul 22 01:42:57.647628 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:42:57.647597 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d6c76ff_846e_41aa_b84d_b1003ca82b6b.slice/crio-ef1415747026142bacc272a2d6bf63edf885e39f7171b60e3a4d28f0f626c9cf WatchSource:0}: Error finding container ef1415747026142bacc272a2d6bf63edf885e39f7171b60e3a4d28f0f626c9cf: Status 404 returned error can't find the container with id ef1415747026142bacc272a2d6bf63edf885e39f7171b60e3a4d28f0f626c9cf Jul 22 01:42:58.389361 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:58.389305 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" event={"ID":"39722719-282b-48fe-be81-2472e3eaaada","Type":"ContainerStarted","Data":"d9081e677116c40ff2c9b789c681caac266c1c6a24fae75ef66b062d18b5b477"} Jul 22 01:42:58.390814 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:58.390786 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" event={"ID":"7cfb6b91-6e4b-460d-a7b3-30134fb115ce","Type":"ContainerStarted","Data":"d7bc26e61af0624720ddedb3345560354bd9531d7bf3fd936cd4134abeb1fd57"} Jul 22 01:42:58.391846 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:58.391826 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" event={"ID":"8d6c76ff-846e-41aa-b84d-b1003ca82b6b","Type":"ContainerStarted","Data":"ef1415747026142bacc272a2d6bf63edf885e39f7171b60e3a4d28f0f626c9cf"} Jul 22 01:42:58.410462 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:58.410409 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" podStartSLOduration=1.320034942 podStartE2EDuration="4.410392258s" totalImagesPullingTime="3.090357316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:42:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:42:58.408580686 +0000 UTC m=+363.237829033" watchObservedRunningTime="2026-07-22 01:42:58.410392258 +0000 UTC m=+363.239640604" Jul 22 01:42:58.427800 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:58.427744 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" podStartSLOduration=2.27557075 podStartE2EDuration="5.427727645s" totalImagesPullingTime="3.152156895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:42:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:42:58.427097102 +0000 UTC m=+363.256345449" watchObservedRunningTime="2026-07-22 01:42:58.427727645 +0000 UTC m=+363.256975990" Jul 22 01:42:59.279760 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:59.279720 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:42:59.280781 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:59.280741 2559 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body= Jul 22 01:42:59.280907 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:59.280808 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" podUID="7cfb6b91-6e4b-460d-a7b3-30134fb115ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" Jul 22 01:42:59.359550 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:59.359513 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:59.364473 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:59.364446 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:59.397241 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:59.397206 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:59.398228 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:59.398209 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sljsh" Jul 22 01:42:59.451777 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:42:59.451723 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg"] Jul 22 01:43:00.280591 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:00.280552 2559 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body= Jul 22 01:43:00.281046 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:00.280622 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" podUID="7cfb6b91-6e4b-460d-a7b3-30134fb115ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" Jul 22 01:43:00.505615 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:00.505576 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 01:43:00.505694 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:00.505649 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 01:43:01.280563 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:01.280524 2559 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body= Jul 22 01:43:01.280770 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:01.280589 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" podUID="7cfb6b91-6e4b-460d-a7b3-30134fb115ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" Jul 22 01:43:01.408018 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:01.407980 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" event={"ID":"8d6c76ff-846e-41aa-b84d-b1003ca82b6b","Type":"ContainerStarted","Data":"ccd33391f4d25b250493951bc137d427d3eaa84c5fd21f2299a57e2ee48fabb4"} Jul 22 01:43:01.408402 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:01.408381 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:43:01.408507 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:01.408484 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" podUID="7cfb6b91-6e4b-460d-a7b3-30134fb115ce" containerName="istio-proxy" containerID="cri-o://d7bc26e61af0624720ddedb3345560354bd9531d7bf3fd936cd4134abeb1fd57" gracePeriod=30 Jul 22 01:43:01.409969 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:01.409939 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" Jul 22 01:43:01.437552 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:01.437511 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5cw2t" podStartSLOduration=2.582042311 podStartE2EDuration="5.437495386s" totalImagesPullingTime="2.855453075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:42:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:43:01.436407759 +0000 UTC m=+366.265656110" watchObservedRunningTime="2026-07-22 01:43:01.437495386 +0000 UTC m=+366.266743734" Jul 22 01:43:06.656707 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.656681 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:43:06.780427 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.780327 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-credential-socket\") pod \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " Jul 22 01:43:06.780427 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.780385 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fctzc\" (UniqueName: \"kubernetes.io/projected/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-kube-api-access-fctzc\") pod \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " Jul 22 01:43:06.780427 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.780419 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-workload-socket\") pod \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " Jul 22 01:43:06.780659 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.780479 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-token\") pod \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " Jul 22 01:43:06.780659 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.780502 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-workload-certs\") pod \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " Jul 22 01:43:06.780659 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.780535 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-data\") pod \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " Jul 22 01:43:06.780659 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.780592 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istiod-ca-cert\") pod \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " Jul 22 01:43:06.780659 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.780625 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-envoy\") pod \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " Jul 22 01:43:06.780659 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.780644 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7cfb6b91-6e4b-460d-a7b3-30134fb115ce" (UID: "7cfb6b91-6e4b-460d-a7b3-30134fb115ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 01:43:06.780659 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.780661 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-podinfo\") pod \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\" (UID: \"7cfb6b91-6e4b-460d-a7b3-30134fb115ce\") " Jul 22 01:43:06.780989 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.780738 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7cfb6b91-6e4b-460d-a7b3-30134fb115ce" (UID: "7cfb6b91-6e4b-460d-a7b3-30134fb115ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 01:43:06.780989 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.780806 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-data" (OuterVolumeSpecName: "istio-data") pod "7cfb6b91-6e4b-460d-a7b3-30134fb115ce" (UID: "7cfb6b91-6e4b-460d-a7b3-30134fb115ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 01:43:06.780989 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.780838 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7cfb6b91-6e4b-460d-a7b3-30134fb115ce" (UID: "7cfb6b91-6e4b-460d-a7b3-30134fb115ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 01:43:06.781118 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.781003 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7cfb6b91-6e4b-460d-a7b3-30134fb115ce" (UID: "7cfb6b91-6e4b-460d-a7b3-30134fb115ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 01:43:06.781118 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.781051 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-credential-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:43:06.781118 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.781070 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-workload-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:43:06.781118 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.781084 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-workload-certs\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:43:06.781118 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.781097 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-data\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:43:06.782838 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.782730 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-token" (OuterVolumeSpecName: "istio-token") pod "7cfb6b91-6e4b-460d-a7b3-30134fb115ce" (UID: "7cfb6b91-6e4b-460d-a7b3-30134fb115ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 01:43:06.782838 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.782803 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7cfb6b91-6e4b-460d-a7b3-30134fb115ce" (UID: "7cfb6b91-6e4b-460d-a7b3-30134fb115ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 01:43:06.783008 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.782904 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-kube-api-access-fctzc" (OuterVolumeSpecName: "kube-api-access-fctzc") pod "7cfb6b91-6e4b-460d-a7b3-30134fb115ce" (UID: "7cfb6b91-6e4b-460d-a7b3-30134fb115ce"). InnerVolumeSpecName "kube-api-access-fctzc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 01:43:06.783008 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.782924 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7cfb6b91-6e4b-460d-a7b3-30134fb115ce" (UID: "7cfb6b91-6e4b-460d-a7b3-30134fb115ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 01:43:06.881738 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.881689 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-podinfo\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:43:06.881738 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.881733 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fctzc\" (UniqueName: \"kubernetes.io/projected/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-kube-api-access-fctzc\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:43:06.881738 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.881742 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-token\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:43:06.881738 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.881753 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istiod-ca-cert\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:43:06.881738 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:06.881761 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cfb6b91-6e4b-460d-a7b3-30134fb115ce-istio-envoy\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 01:43:07.439756 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:07.439716 2559 generic.go:356] "Generic (PLEG): container finished" podID="7cfb6b91-6e4b-460d-a7b3-30134fb115ce" containerID="d7bc26e61af0624720ddedb3345560354bd9531d7bf3fd936cd4134abeb1fd57" exitCode=0 Jul 22 01:43:07.439950 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:07.439765 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" Jul 22 01:43:07.439950 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:07.439806 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" event={"ID":"7cfb6b91-6e4b-460d-a7b3-30134fb115ce","Type":"ContainerDied","Data":"d7bc26e61af0624720ddedb3345560354bd9531d7bf3fd936cd4134abeb1fd57"} Jul 22 01:43:07.439950 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:07.439874 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg" event={"ID":"7cfb6b91-6e4b-460d-a7b3-30134fb115ce","Type":"ContainerDied","Data":"d7455aa07ce9801a167c62711023c8cd640ec17e4193b7c5de42afbf387a6bc2"} Jul 22 01:43:07.439950 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:07.439904 2559 scope.go:117] "RemoveContainer" containerID="d7bc26e61af0624720ddedb3345560354bd9531d7bf3fd936cd4134abeb1fd57" Jul 22 01:43:07.449723 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:07.449708 2559 scope.go:117] "RemoveContainer" containerID="d7bc26e61af0624720ddedb3345560354bd9531d7bf3fd936cd4134abeb1fd57" Jul 22 01:43:07.450000 ip-10-0-131-80 kubenswrapper[2559]: E0722 01:43:07.449981 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7bc26e61af0624720ddedb3345560354bd9531d7bf3fd936cd4134abeb1fd57\": container with ID starting with d7bc26e61af0624720ddedb3345560354bd9531d7bf3fd936cd4134abeb1fd57 not found: ID does not exist" containerID="d7bc26e61af0624720ddedb3345560354bd9531d7bf3fd936cd4134abeb1fd57" Jul 22 01:43:07.450045 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:07.450010 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7bc26e61af0624720ddedb3345560354bd9531d7bf3fd936cd4134abeb1fd57"} err="failed to get container status \"d7bc26e61af0624720ddedb3345560354bd9531d7bf3fd936cd4134abeb1fd57\": rpc error: code = NotFound desc = could not find container \"d7bc26e61af0624720ddedb3345560354bd9531d7bf3fd936cd4134abeb1fd57\": container with ID starting with d7bc26e61af0624720ddedb3345560354bd9531d7bf3fd936cd4134abeb1fd57 not found: ID does not exist" Jul 22 01:43:07.463991 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:07.463938 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg"] Jul 22 01:43:07.468157 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:07.468132 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-76xlg"] Jul 22 01:43:07.625641 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:43:07.625607 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cfb6b91-6e4b-460d-a7b3-30134fb115ce" path="/var/lib/kubelet/pods/7cfb6b91-6e4b-460d-a7b3-30134fb115ce/volumes" Jul 22 01:45:06.329047 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.329009 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cfb8c6644-9fc82"] Jul 22 01:45:06.329662 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.329639 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cfb6b91-6e4b-460d-a7b3-30134fb115ce" containerName="istio-proxy" Jul 22 01:45:06.329741 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.329667 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cfb6b91-6e4b-460d-a7b3-30134fb115ce" containerName="istio-proxy" Jul 22 01:45:06.329796 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.329788 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cfb6b91-6e4b-460d-a7b3-30134fb115ce" containerName="istio-proxy" Jul 22 01:45:06.332391 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.332368 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.335129 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.335105 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 01:45:06.336123 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.335971 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 22 01:45:06.336272 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.336245 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kl75w\"" Jul 22 01:45:06.336404 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.336366 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 01:45:06.336475 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.336454 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 01:45:06.336540 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.336483 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 01:45:06.336696 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.336681 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 22 01:45:06.336696 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.336691 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 01:45:06.342201 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.340896 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 22 01:45:06.342391 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.342368 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ef1db5a-7aba-4df2-822a-495acb593e8b-trusted-ca-bundle\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.342510 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.342407 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ef1db5a-7aba-4df2-822a-495acb593e8b-service-ca\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.342510 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.342437 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ef1db5a-7aba-4df2-822a-495acb593e8b-console-oauth-config\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.342618 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.342523 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkvpn\" (UniqueName: \"kubernetes.io/projected/6ef1db5a-7aba-4df2-822a-495acb593e8b-kube-api-access-lkvpn\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.342618 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.342570 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ef1db5a-7aba-4df2-822a-495acb593e8b-console-serving-cert\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.342618 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.342591 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ef1db5a-7aba-4df2-822a-495acb593e8b-console-config\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.342618 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.342611 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ef1db5a-7aba-4df2-822a-495acb593e8b-oauth-serving-cert\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.343560 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.343537 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cfb8c6644-9fc82"] Jul 22 01:45:06.443629 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.443597 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ef1db5a-7aba-4df2-822a-495acb593e8b-console-serving-cert\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.443629 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.443638 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ef1db5a-7aba-4df2-822a-495acb593e8b-console-config\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.443890 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.443661 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ef1db5a-7aba-4df2-822a-495acb593e8b-oauth-serving-cert\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.443890 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.443735 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ef1db5a-7aba-4df2-822a-495acb593e8b-trusted-ca-bundle\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.443890 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.443755 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ef1db5a-7aba-4df2-822a-495acb593e8b-service-ca\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.443890 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.443779 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ef1db5a-7aba-4df2-822a-495acb593e8b-console-oauth-config\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.443890 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.443841 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkvpn\" (UniqueName: \"kubernetes.io/projected/6ef1db5a-7aba-4df2-822a-495acb593e8b-kube-api-access-lkvpn\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.444504 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.444475 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ef1db5a-7aba-4df2-822a-495acb593e8b-oauth-serving-cert\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.444504 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.444491 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ef1db5a-7aba-4df2-822a-495acb593e8b-service-ca\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.444662 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.444493 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ef1db5a-7aba-4df2-822a-495acb593e8b-console-config\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.444789 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.444767 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ef1db5a-7aba-4df2-822a-495acb593e8b-trusted-ca-bundle\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.446156 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.446139 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ef1db5a-7aba-4df2-822a-495acb593e8b-console-serving-cert\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.446289 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.446268 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ef1db5a-7aba-4df2-822a-495acb593e8b-console-oauth-config\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.453139 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.453113 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkvpn\" (UniqueName: \"kubernetes.io/projected/6ef1db5a-7aba-4df2-822a-495acb593e8b-kube-api-access-lkvpn\") pod \"console-6cfb8c6644-9fc82\" (UID: \"6ef1db5a-7aba-4df2-822a-495acb593e8b\") " pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.645386 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.645282 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:06.785289 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.785246 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cfb8c6644-9fc82"] Jul 22 01:45:06.788322 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:45:06.788294 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ef1db5a_7aba_4df2_822a_495acb593e8b.slice/crio-a773adcbf7068e68c306ebdde5c0f52dd1b0a2e7364c65d7b1029779841bf013 WatchSource:0}: Error finding container a773adcbf7068e68c306ebdde5c0f52dd1b0a2e7364c65d7b1029779841bf013: Status 404 returned error can't find the container with id a773adcbf7068e68c306ebdde5c0f52dd1b0a2e7364c65d7b1029779841bf013 Jul 22 01:45:06.979046 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.978957 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cfb8c6644-9fc82" event={"ID":"6ef1db5a-7aba-4df2-822a-495acb593e8b","Type":"ContainerStarted","Data":"46d228b390b78f2bb6f51b9e325d17a5c80550fe7c04f36af363e9c44ed62bb2"} Jul 22 01:45:06.979046 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.979011 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cfb8c6644-9fc82" event={"ID":"6ef1db5a-7aba-4df2-822a-495acb593e8b","Type":"ContainerStarted","Data":"a773adcbf7068e68c306ebdde5c0f52dd1b0a2e7364c65d7b1029779841bf013"} Jul 22 01:45:06.999564 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:06.999515 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cfb8c6644-9fc82" podStartSLOduration=0.999499242 podStartE2EDuration="999.499242ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:45:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:45:06.998401456 +0000 UTC m=+491.827649806" watchObservedRunningTime="2026-07-22 01:45:06.999499242 +0000 UTC m=+491.828747589" Jul 22 01:45:12.637091 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.637054 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl"] Jul 22 01:45:12.640534 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.640517 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.642824 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.642799 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-8kvr7\"" Jul 22 01:45:12.655975 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.655950 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl"] Jul 22 01:45:12.692570 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.692540 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbxkl\" (UniqueName: \"kubernetes.io/projected/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-kube-api-access-xbxkl\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.692723 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.692582 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.692723 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.692617 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.692723 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.692646 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.692723 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.692664 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.692723 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.692681 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.692978 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.692747 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.692978 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.692787 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.692978 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.692852 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.793800 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.793764 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.793969 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.793817 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.793969 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.793859 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.793969 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.793905 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbxkl\" (UniqueName: \"kubernetes.io/projected/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-kube-api-access-xbxkl\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.793969 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.793938 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.793969 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.793962 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.794233 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.794005 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.794233 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.794038 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.794233 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.794070 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.794428 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.794336 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.794494 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.794444 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.794494 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.794472 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.794733 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.794710 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.794803 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.794782 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.796152 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.796132 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.796273 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.796256 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.807188 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.807151 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.807760 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.807736 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbxkl\" (UniqueName: \"kubernetes.io/projected/9ccf5793-b290-4f72-a31d-a5d13b6dbd3e-kube-api-access-xbxkl\") pod \"maas-default-gateway-openshift-default-56654dcf7-wrrzl\" (UID: \"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:12.950840 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:12.950751 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:13.086883 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:13.086858 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl"] Jul 22 01:45:13.089015 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:45:13.088986 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ccf5793_b290_4f72_a31d_a5d13b6dbd3e.slice/crio-c916ea1e18e690b0bf430a34a6a7558a88432c453ac36b5a38ce60772a51ec9e WatchSource:0}: Error finding container c916ea1e18e690b0bf430a34a6a7558a88432c453ac36b5a38ce60772a51ec9e: Status 404 returned error can't find the container with id c916ea1e18e690b0bf430a34a6a7558a88432c453ac36b5a38ce60772a51ec9e Jul 22 01:45:13.091085 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:13.091051 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 01:45:13.091183 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:13.091119 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 01:45:13.091183 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:13.091145 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 01:45:14.014670 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:14.014631 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" event={"ID":"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e","Type":"ContainerStarted","Data":"cf166c38d53c227ecbdc6b3f4a5bab52d6fcc162bb30ca7a4715fcd8f2c7a522"} Jul 22 01:45:14.015041 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:14.014690 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" event={"ID":"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e","Type":"ContainerStarted","Data":"c916ea1e18e690b0bf430a34a6a7558a88432c453ac36b5a38ce60772a51ec9e"} Jul 22 01:45:14.035979 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:14.035914 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" podStartSLOduration=2.035894306 podStartE2EDuration="2.035894306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:45:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:45:14.034610945 +0000 UTC m=+498.863859294" watchObservedRunningTime="2026-07-22 01:45:14.035894306 +0000 UTC m=+498.865142652" Jul 22 01:45:14.951148 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:14.951109 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:14.956030 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:14.956006 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:15.018832 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:15.018795 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:15.019794 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:15.019772 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:45:16.646360 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:16.646307 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:16.646790 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:16.646388 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:16.651096 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:16.651071 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:45:17.031918 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:45:17.031839 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6cfb8c6644-9fc82" Jul 22 01:46:51.473849 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:51.473814 2559 generic.go:356] "Generic (PLEG): container finished" podID="9ccf5793-b290-4f72-a31d-a5d13b6dbd3e" containerID="cf166c38d53c227ecbdc6b3f4a5bab52d6fcc162bb30ca7a4715fcd8f2c7a522" exitCode=0 Jul 22 01:46:51.474220 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:51.473909 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" event={"ID":"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e","Type":"ContainerDied","Data":"cf166c38d53c227ecbdc6b3f4a5bab52d6fcc162bb30ca7a4715fcd8f2c7a522"} Jul 22 01:46:51.474294 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:51.474281 2559 scope.go:117] "RemoveContainer" containerID="cf166c38d53c227ecbdc6b3f4a5bab52d6fcc162bb30ca7a4715fcd8f2c7a522" Jul 22 01:46:51.475254 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:51.475226 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 01:46:51.475362 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:51.475287 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 01:46:51.475362 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:51.475322 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 01:46:52.481173 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:52.481135 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" event={"ID":"9ccf5793-b290-4f72-a31d-a5d13b6dbd3e","Type":"ContainerStarted","Data":"01074c6d56eea9a1d311fa54e12550147fec18e7b9bbc5f60cd7dc169914955b"} Jul 22 01:46:52.951183 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:52.951142 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:46:52.951469 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:52.951441 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wrrzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 22 01:46:52.951597 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:52.951501 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" podUID="9ccf5793-b290-4f72-a31d-a5d13b6dbd3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 22 01:46:53.951327 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:53.951286 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wrrzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 22 01:46:53.951849 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:53.951365 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" podUID="9ccf5793-b290-4f72-a31d-a5d13b6dbd3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 22 01:46:54.951775 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:54.951723 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wrrzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 22 01:46:54.952417 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:54.952383 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" podUID="9ccf5793-b290-4f72-a31d-a5d13b6dbd3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 22 01:46:55.515845 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:55.515815 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 01:46:55.517577 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:55.517554 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 01:46:55.520411 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:55.520394 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 01:46:55.522075 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:55.522054 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 01:46:55.952006 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:55.951971 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-wrrzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 22 01:46:55.952626 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:55.952584 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" podUID="9ccf5793-b290-4f72-a31d-a5d13b6dbd3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 22 01:46:56.954988 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:56.954962 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:46:56.955422 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:56.955235 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:46:56.955854 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:56.955838 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-wrrzl" Jul 22 01:46:59.689002 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.688969 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7"] Jul 22 01:46:59.692623 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.692604 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:46:59.695875 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.695851 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 22 01:46:59.695875 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.695880 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 22 01:46:59.696079 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.695854 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-v8lsh\"" Jul 22 01:46:59.696079 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.695860 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 22 01:46:59.704079 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.704052 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7"] Jul 22 01:46:59.746238 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.746200 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df704e77-dd0a-4766-afcc-8aa69afc532f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7\" (UID: \"df704e77-dd0a-4766-afcc-8aa69afc532f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:46:59.746238 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.746231 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df704e77-dd0a-4766-afcc-8aa69afc532f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7\" (UID: \"df704e77-dd0a-4766-afcc-8aa69afc532f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:46:59.746453 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.746260 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df704e77-dd0a-4766-afcc-8aa69afc532f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7\" (UID: \"df704e77-dd0a-4766-afcc-8aa69afc532f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:46:59.746453 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.746377 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df704e77-dd0a-4766-afcc-8aa69afc532f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7\" (UID: \"df704e77-dd0a-4766-afcc-8aa69afc532f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:46:59.746453 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.746404 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df704e77-dd0a-4766-afcc-8aa69afc532f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7\" (UID: \"df704e77-dd0a-4766-afcc-8aa69afc532f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:46:59.746453 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.746441 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2grw9\" (UniqueName: \"kubernetes.io/projected/df704e77-dd0a-4766-afcc-8aa69afc532f-kube-api-access-2grw9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7\" (UID: \"df704e77-dd0a-4766-afcc-8aa69afc532f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:46:59.847738 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.847704 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2grw9\" (UniqueName: \"kubernetes.io/projected/df704e77-dd0a-4766-afcc-8aa69afc532f-kube-api-access-2grw9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7\" (UID: \"df704e77-dd0a-4766-afcc-8aa69afc532f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:46:59.847896 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.847748 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df704e77-dd0a-4766-afcc-8aa69afc532f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7\" (UID: \"df704e77-dd0a-4766-afcc-8aa69afc532f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:46:59.847896 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.847766 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df704e77-dd0a-4766-afcc-8aa69afc532f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7\" (UID: \"df704e77-dd0a-4766-afcc-8aa69afc532f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:46:59.847896 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.847813 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df704e77-dd0a-4766-afcc-8aa69afc532f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7\" (UID: \"df704e77-dd0a-4766-afcc-8aa69afc532f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:46:59.847896 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.847891 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df704e77-dd0a-4766-afcc-8aa69afc532f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7\" (UID: \"df704e77-dd0a-4766-afcc-8aa69afc532f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:46:59.848085 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.847917 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df704e77-dd0a-4766-afcc-8aa69afc532f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7\" (UID: \"df704e77-dd0a-4766-afcc-8aa69afc532f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:46:59.848238 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.848209 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df704e77-dd0a-4766-afcc-8aa69afc532f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7\" (UID: \"df704e77-dd0a-4766-afcc-8aa69afc532f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:46:59.848430 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.848253 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df704e77-dd0a-4766-afcc-8aa69afc532f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7\" (UID: \"df704e77-dd0a-4766-afcc-8aa69afc532f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:46:59.848430 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.848375 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df704e77-dd0a-4766-afcc-8aa69afc532f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7\" (UID: \"df704e77-dd0a-4766-afcc-8aa69afc532f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:46:59.851096 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.851072 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df704e77-dd0a-4766-afcc-8aa69afc532f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7\" (UID: \"df704e77-dd0a-4766-afcc-8aa69afc532f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:46:59.851286 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.851081 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df704e77-dd0a-4766-afcc-8aa69afc532f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7\" (UID: \"df704e77-dd0a-4766-afcc-8aa69afc532f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:46:59.856845 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:46:59.856825 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2grw9\" (UniqueName: \"kubernetes.io/projected/df704e77-dd0a-4766-afcc-8aa69afc532f-kube-api-access-2grw9\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7\" (UID: \"df704e77-dd0a-4766-afcc-8aa69afc532f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:47:00.003544 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:00.003453 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:47:00.143599 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:00.143566 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7"] Jul 22 01:47:00.145373 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:47:00.145337 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf704e77_dd0a_4766_afcc_8aa69afc532f.slice/crio-a64569550cecb1a6fab3e9109af4224071f8b3ccead510826686a78634a61dc2 WatchSource:0}: Error finding container a64569550cecb1a6fab3e9109af4224071f8b3ccead510826686a78634a61dc2: Status 404 returned error can't find the container with id a64569550cecb1a6fab3e9109af4224071f8b3ccead510826686a78634a61dc2 Jul 22 01:47:00.520838 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:00.520801 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" event={"ID":"df704e77-dd0a-4766-afcc-8aa69afc532f","Type":"ContainerStarted","Data":"a64569550cecb1a6fab3e9109af4224071f8b3ccead510826686a78634a61dc2"} Jul 22 01:47:01.529473 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:01.529380 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" event={"ID":"df704e77-dd0a-4766-afcc-8aa69afc532f","Type":"ContainerStarted","Data":"b9a49c9927458769df1b0e844ae71a0ac2f627809d3c5eccd62d27a8f03ede04"} Jul 22 01:47:01.552868 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:01.552821 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" podStartSLOduration=1.571923851 podStartE2EDuration="2.55280464s" totalImagesPullingTime="980.880789ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:47:01.551760448 +0000 UTC m=+606.381008803" watchObservedRunningTime="2026-07-22 01:47:01.55280464 +0000 UTC m=+606.382052989" Jul 22 01:47:10.003868 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:10.003830 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:47:10.004396 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:10.003883 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:47:10.016317 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:10.016289 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:47:10.581992 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:10.581962 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7" Jul 22 01:47:24.989838 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:24.989803 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p"] Jul 22 01:47:24.993268 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:24.993246 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:24.995654 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:24.995625 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 22 01:47:25.004116 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.004090 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p"] Jul 22 01:47:25.084358 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.084305 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j7f9\" (UniqueName: \"kubernetes.io/projected/7f5131ff-c8f7-448c-b051-23dc404eff6b-kube-api-access-9j7f9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p\" (UID: \"7f5131ff-c8f7-448c-b051-23dc404eff6b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:25.084511 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.084401 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f5131ff-c8f7-448c-b051-23dc404eff6b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p\" (UID: \"7f5131ff-c8f7-448c-b051-23dc404eff6b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:25.084571 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.084541 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f5131ff-c8f7-448c-b051-23dc404eff6b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p\" (UID: \"7f5131ff-c8f7-448c-b051-23dc404eff6b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:25.084620 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.084604 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f5131ff-c8f7-448c-b051-23dc404eff6b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p\" (UID: \"7f5131ff-c8f7-448c-b051-23dc404eff6b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:25.084660 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.084635 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f5131ff-c8f7-448c-b051-23dc404eff6b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p\" (UID: \"7f5131ff-c8f7-448c-b051-23dc404eff6b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:25.084704 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.084661 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f5131ff-c8f7-448c-b051-23dc404eff6b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p\" (UID: \"7f5131ff-c8f7-448c-b051-23dc404eff6b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:25.185394 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.185337 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f5131ff-c8f7-448c-b051-23dc404eff6b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p\" (UID: \"7f5131ff-c8f7-448c-b051-23dc404eff6b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:25.185394 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.185398 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f5131ff-c8f7-448c-b051-23dc404eff6b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p\" (UID: \"7f5131ff-c8f7-448c-b051-23dc404eff6b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:25.185631 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.185420 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f5131ff-c8f7-448c-b051-23dc404eff6b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p\" (UID: \"7f5131ff-c8f7-448c-b051-23dc404eff6b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:25.185631 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.185451 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j7f9\" (UniqueName: \"kubernetes.io/projected/7f5131ff-c8f7-448c-b051-23dc404eff6b-kube-api-access-9j7f9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p\" (UID: \"7f5131ff-c8f7-448c-b051-23dc404eff6b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:25.185631 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.185555 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f5131ff-c8f7-448c-b051-23dc404eff6b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p\" (UID: \"7f5131ff-c8f7-448c-b051-23dc404eff6b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:25.185799 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.185665 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f5131ff-c8f7-448c-b051-23dc404eff6b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p\" (UID: \"7f5131ff-c8f7-448c-b051-23dc404eff6b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:25.185893 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.185869 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f5131ff-c8f7-448c-b051-23dc404eff6b-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p\" (UID: \"7f5131ff-c8f7-448c-b051-23dc404eff6b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:25.185951 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.185900 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f5131ff-c8f7-448c-b051-23dc404eff6b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p\" (UID: \"7f5131ff-c8f7-448c-b051-23dc404eff6b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:25.186051 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.186035 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f5131ff-c8f7-448c-b051-23dc404eff6b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p\" (UID: \"7f5131ff-c8f7-448c-b051-23dc404eff6b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:25.187628 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.187603 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f5131ff-c8f7-448c-b051-23dc404eff6b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p\" (UID: \"7f5131ff-c8f7-448c-b051-23dc404eff6b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:25.187971 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.187953 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f5131ff-c8f7-448c-b051-23dc404eff6b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p\" (UID: \"7f5131ff-c8f7-448c-b051-23dc404eff6b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:25.194691 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.194672 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j7f9\" (UniqueName: \"kubernetes.io/projected/7f5131ff-c8f7-448c-b051-23dc404eff6b-kube-api-access-9j7f9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p\" (UID: \"7f5131ff-c8f7-448c-b051-23dc404eff6b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:25.306252 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.306162 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:25.438580 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.438531 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p"] Jul 22 01:47:25.441010 ip-10-0-131-80 kubenswrapper[2559]: W0722 01:47:25.440978 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f5131ff_c8f7_448c_b051_23dc404eff6b.slice/crio-663b36c52487881b708fd8c9b78bd1a87e8c62d642b1d005b2b1a8ebe34a5e69 WatchSource:0}: Error finding container 663b36c52487881b708fd8c9b78bd1a87e8c62d642b1d005b2b1a8ebe34a5e69: Status 404 returned error can't find the container with id 663b36c52487881b708fd8c9b78bd1a87e8c62d642b1d005b2b1a8ebe34a5e69 Jul 22 01:47:25.442387 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.442366 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 01:47:25.639145 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:25.639107 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" event={"ID":"7f5131ff-c8f7-448c-b051-23dc404eff6b","Type":"ContainerStarted","Data":"663b36c52487881b708fd8c9b78bd1a87e8c62d642b1d005b2b1a8ebe34a5e69"} Jul 22 01:47:26.645464 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:26.645419 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" event={"ID":"7f5131ff-c8f7-448c-b051-23dc404eff6b","Type":"ContainerStarted","Data":"1afd83f801142642429512b3d2b7714f753027370ebc90dfe053bb7a63176170"} Jul 22 01:47:26.665731 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:26.665676 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" podStartSLOduration=2.440982332 podStartE2EDuration="2.665657424s" totalImagesPullingTime="224.675092ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:47:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:47:26.664754536 +0000 UTC m=+631.494002884" watchObservedRunningTime="2026-07-22 01:47:26.665657424 +0000 UTC m=+631.494905782" Jul 22 01:47:35.307277 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:35.307239 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:35.307684 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:35.307303 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:35.319879 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:35.319853 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:47:35.695101 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:47:35.695074 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p" Jul 22 01:51:55.545550 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:51:55.545513 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 01:51:55.548089 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:51:55.545832 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 01:51:55.550145 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:51:55.550124 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 01:51:55.550917 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:51:55.550892 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 01:56:55.578104 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:56:55.578077 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 01:56:55.580365 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:56:55.579025 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 01:56:55.582507 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:56:55.582492 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 01:56:55.583389 ip-10-0-131-80 kubenswrapper[2559]: I0722 01:56:55.583372 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 02:00:00.159727 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:00:00.159698 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744760-7qpbv"] Jul 22 02:00:00.163170 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:00:00.163152 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744760-7qpbv" Jul 22 02:00:00.165974 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:00:00.165956 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cqdw8\"" Jul 22 02:00:00.172405 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:00:00.172390 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744760-7qpbv" Jul 22 02:00:00.186562 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:00:00.186542 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744760-7qpbv"] Jul 22 02:00:00.311795 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:00:00.311770 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744760-7qpbv"] Jul 22 02:00:00.313508 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:00:00.313480 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7925c8d2_4273_4b98_bfd2_5005d27454c3.slice/crio-c7ebf4af3dc104c1945d39960a417d1ae119615421c1f079ef503ae3dd4baa47 WatchSource:0}: Error finding container c7ebf4af3dc104c1945d39960a417d1ae119615421c1f079ef503ae3dd4baa47: Status 404 returned error can't find the container with id c7ebf4af3dc104c1945d39960a417d1ae119615421c1f079ef503ae3dd4baa47 Jul 22 02:00:00.315489 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:00:00.315472 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:00:01.094063 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:00:01.094028 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744760-7qpbv" event={"ID":"7925c8d2-4273-4b98-bfd2-5005d27454c3","Type":"ContainerStarted","Data":"c7ebf4af3dc104c1945d39960a417d1ae119615421c1f079ef503ae3dd4baa47"} Jul 22 02:00:13.155939 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:00:13.155862 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744760-7qpbv" event={"ID":"7925c8d2-4273-4b98-bfd2-5005d27454c3","Type":"ContainerStarted","Data":"aea18a8ec87b8be1f6dbe87f25f639df50d207430adb8586ed416b1a6b05433b"} Jul 22 02:00:13.181645 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:00:13.181601 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744760-7qpbv" podStartSLOduration=0.963518925 podStartE2EDuration="13.181587639s" totalImagesPullingTime="12.218068714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:00:13.180402306 +0000 UTC m=+1398.009650654" watchObservedRunningTime="2026-07-22 02:00:13.181587639 +0000 UTC m=+1398.010835986" Jul 22 02:00:14.161943 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:00:14.161906 2559 generic.go:356] "Generic (PLEG): container finished" podID="7925c8d2-4273-4b98-bfd2-5005d27454c3" containerID="aea18a8ec87b8be1f6dbe87f25f639df50d207430adb8586ed416b1a6b05433b" exitCode=0 Jul 22 02:00:14.162315 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:00:14.161984 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744760-7qpbv" event={"ID":"7925c8d2-4273-4b98-bfd2-5005d27454c3","Type":"ContainerDied","Data":"aea18a8ec87b8be1f6dbe87f25f639df50d207430adb8586ed416b1a6b05433b"} Jul 22 02:00:15.300443 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:00:15.300416 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744760-7qpbv" Jul 22 02:00:16.174486 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:00:16.174442 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744760-7qpbv" event={"ID":"7925c8d2-4273-4b98-bfd2-5005d27454c3","Type":"ContainerDied","Data":"c7ebf4af3dc104c1945d39960a417d1ae119615421c1f079ef503ae3dd4baa47"} Jul 22 02:00:16.174722 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:00:16.174506 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c7ebf4af3dc104c1945d39960a417d1ae119615421c1f079ef503ae3dd4baa47" Jul 22 02:00:16.174722 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:00:16.174467 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744760-7qpbv" Jul 22 02:01:55.606636 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:01:55.606529 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 02:01:55.610519 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:01:55.608886 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 02:01:55.611129 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:01:55.611110 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 02:01:55.613242 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:01:55.613224 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 02:04:24.363407 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.363370 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2"] Jul 22 02:04:24.363881 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.363750 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7925c8d2-4273-4b98-bfd2-5005d27454c3" containerName="cleanup" Jul 22 02:04:24.363881 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.363765 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7925c8d2-4273-4b98-bfd2-5005d27454c3" containerName="cleanup" Jul 22 02:04:24.363881 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.363844 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7925c8d2-4273-4b98-bfd2-5005d27454c3" containerName="cleanup" Jul 22 02:04:24.367121 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.367102 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.369410 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.369390 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f2f0e842-openshift-default-dockercfg-sc89r\"" Jul 22 02:04:24.386321 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.386297 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2"] Jul 22 02:04:24.490070 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.490030 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-workload-socket\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.490070 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.490073 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-data\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.490300 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.490108 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-credential-socket\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.490300 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.490140 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-token\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.490300 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.490160 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98m7z\" (UniqueName: \"kubernetes.io/projected/e93613b6-5403-4b04-8300-34ebe1c3b5b3-kube-api-access-98m7z\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.490300 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.490239 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-workload-certs\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.490300 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.490277 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istiod-ca-cert\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.490300 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.490296 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-envoy\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.490536 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.490337 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-podinfo\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.591004 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.590969 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-workload-socket\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.591004 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.591003 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-data\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.591287 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.591030 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-credential-socket\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.591287 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.591061 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-token\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.591287 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.591083 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98m7z\" (UniqueName: \"kubernetes.io/projected/e93613b6-5403-4b04-8300-34ebe1c3b5b3-kube-api-access-98m7z\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.591287 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.591116 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-workload-certs\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.591287 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.591147 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istiod-ca-cert\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.591287 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.591204 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-envoy\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.591287 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.591284 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-podinfo\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.591616 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.591574 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-data\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.591714 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.591686 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-credential-socket\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.591803 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.591721 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-workload-socket\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.591860 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.591828 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-workload-certs\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.591907 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.591855 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istiod-ca-cert\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.593638 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.593616 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-envoy\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.593741 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.593727 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-podinfo\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.599699 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.599679 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-token\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.599909 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.599888 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98m7z\" (UniqueName: \"kubernetes.io/projected/e93613b6-5403-4b04-8300-34ebe1c3b5b3-kube-api-access-98m7z\") pod \"e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.677915 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.677818 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:24.814377 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.814330 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2"] Jul 22 02:04:24.816553 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:04:24.816523 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode93613b6_5403_4b04_8300_34ebe1c3b5b3.slice/crio-8115edea4e7c96bb32808de62dd3a15c8ea5c4520b84738b939571973f977bd6 WatchSource:0}: Error finding container 8115edea4e7c96bb32808de62dd3a15c8ea5c4520b84738b939571973f977bd6: Status 404 returned error can't find the container with id 8115edea4e7c96bb32808de62dd3a15c8ea5c4520b84738b939571973f977bd6 Jul 22 02:04:24.818531 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.818495 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:04:24.818628 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.818560 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:04:24.818628 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:24.818589 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:04:25.351472 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:25.351440 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" event={"ID":"e93613b6-5403-4b04-8300-34ebe1c3b5b3","Type":"ContainerStarted","Data":"b5cdc1d6aff7e0ea42c90b43bfd922c26408b240af52ab540d6fb91a76a4221f"} Jul 22 02:04:25.351650 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:25.351488 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" event={"ID":"e93613b6-5403-4b04-8300-34ebe1c3b5b3","Type":"ContainerStarted","Data":"8115edea4e7c96bb32808de62dd3a15c8ea5c4520b84738b939571973f977bd6"} Jul 22 02:04:25.379658 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:25.379607 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" podStartSLOduration=1.379590099 podStartE2EDuration="1.379590099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:04:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:04:25.378251602 +0000 UTC m=+1650.207499966" watchObservedRunningTime="2026-07-22 02:04:25.379590099 +0000 UTC m=+1650.208838446" Jul 22 02:04:25.678183 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:25.678096 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:25.683114 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:25.683089 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:26.356484 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:26.356454 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:26.357415 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:26.357394 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:42.397510 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:42.397475 2559 patch_prober.go:28] interesting pod/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 02:04:42.397900 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:42.397532 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" podUID="e93613b6-5403-4b04-8300-34ebe1c3b5b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 02:04:43.438495 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:43.438463 2559 generic.go:356] "Generic (PLEG): container finished" podID="e93613b6-5403-4b04-8300-34ebe1c3b5b3" containerID="b5cdc1d6aff7e0ea42c90b43bfd922c26408b240af52ab540d6fb91a76a4221f" exitCode=0 Jul 22 02:04:43.438933 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:43.438536 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" event={"ID":"e93613b6-5403-4b04-8300-34ebe1c3b5b3","Type":"ContainerDied","Data":"b5cdc1d6aff7e0ea42c90b43bfd922c26408b240af52ab540d6fb91a76a4221f"} Jul 22 02:04:43.438933 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:43.438866 2559 scope.go:117] "RemoveContainer" containerID="b5cdc1d6aff7e0ea42c90b43bfd922c26408b240af52ab540d6fb91a76a4221f" Jul 22 02:04:43.439788 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:43.439760 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:04:43.439886 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:43.439826 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:04:43.439886 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:43.439858 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:04:44.445254 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:44.445224 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" event={"ID":"e93613b6-5403-4b04-8300-34ebe1c3b5b3","Type":"ContainerStarted","Data":"00d989f9224ca2e8d718a6155ddce4252bd903468df80fb1baca818e6295e113"} Jul 22 02:04:44.678419 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:44.678386 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:44.678587 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:44.678573 2559 patch_prober.go:28] interesting pod/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 22 02:04:44.678649 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:44.678627 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" podUID="e93613b6-5403-4b04-8300-34ebe1c3b5b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 22 02:04:45.678843 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:45.678809 2559 patch_prober.go:28] interesting pod/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 22 02:04:45.679215 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:45.678860 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" podUID="e93613b6-5403-4b04-8300-34ebe1c3b5b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 22 02:04:46.678527 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:46.678494 2559 patch_prober.go:28] interesting pod/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 22 02:04:46.678722 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:46.678562 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" podUID="e93613b6-5403-4b04-8300-34ebe1c3b5b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 22 02:04:47.678685 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:47.678645 2559 patch_prober.go:28] interesting pod/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 22 02:04:47.679078 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:47.678713 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" podUID="e93613b6-5403-4b04-8300-34ebe1c3b5b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 22 02:04:48.678624 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:48.678582 2559 patch_prober.go:28] interesting pod/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 22 02:04:48.678800 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:48.678666 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" podUID="e93613b6-5403-4b04-8300-34ebe1c3b5b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 22 02:04:49.681902 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:49.681864 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:49.682400 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:49.682237 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:49.682994 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:49.682976 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:04:52.438951 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:52.438913 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99"] Jul 22 02:04:52.442708 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:52.442690 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99" Jul 22 02:04:52.445039 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:52.445012 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 02:04:52.445148 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:52.445095 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cqdw8\"" Jul 22 02:04:52.449067 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:52.449046 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99"] Jul 22 02:04:52.542306 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:52.542277 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15deff07-dad1-4b97-b1ec-fd653636c7c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99\" (UID: \"15deff07-dad1-4b97-b1ec-fd653636c7c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99" Jul 22 02:04:52.643328 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:52.643296 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15deff07-dad1-4b97-b1ec-fd653636c7c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99\" (UID: \"15deff07-dad1-4b97-b1ec-fd653636c7c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99" Jul 22 02:04:52.643886 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:52.643863 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15deff07-dad1-4b97-b1ec-fd653636c7c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99\" (UID: \"15deff07-dad1-4b97-b1ec-fd653636c7c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99" Jul 22 02:04:52.754922 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:52.754869 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99" Jul 22 02:04:52.884504 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:52.884472 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99"] Jul 22 02:04:52.887644 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:04:52.887619 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15deff07_dad1_4b97_b1ec_fd653636c7c8.slice/crio-bf75e7eadcaa5ebd78d843951a9847d9ea4dff4731b9816953919a5004453a60 WatchSource:0}: Error finding container bf75e7eadcaa5ebd78d843951a9847d9ea4dff4731b9816953919a5004453a60: Status 404 returned error can't find the container with id bf75e7eadcaa5ebd78d843951a9847d9ea4dff4731b9816953919a5004453a60 Jul 22 02:04:53.492248 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:53.492163 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99" event={"ID":"15deff07-dad1-4b97-b1ec-fd653636c7c8","Type":"ContainerStarted","Data":"e40010ca7f3389080dc954dd2a7453583651a42ccd5894987bc2b302a9869f81"} Jul 22 02:04:53.492248 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:53.492223 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99" event={"ID":"15deff07-dad1-4b97-b1ec-fd653636c7c8","Type":"ContainerStarted","Data":"bf75e7eadcaa5ebd78d843951a9847d9ea4dff4731b9816953919a5004453a60"} Jul 22 02:04:53.509598 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:53.509552 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99" podStartSLOduration=1.5095267159999999 podStartE2EDuration="1.509526716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:04:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:04:53.508673379 +0000 UTC m=+1678.337921727" watchObservedRunningTime="2026-07-22 02:04:53.509526716 +0000 UTC m=+1678.338775065" Jul 22 02:04:54.498370 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:54.498321 2559 generic.go:356] "Generic (PLEG): container finished" podID="15deff07-dad1-4b97-b1ec-fd653636c7c8" containerID="e40010ca7f3389080dc954dd2a7453583651a42ccd5894987bc2b302a9869f81" exitCode=0 Jul 22 02:04:54.498767 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:54.498388 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99" event={"ID":"15deff07-dad1-4b97-b1ec-fd653636c7c8","Type":"ContainerDied","Data":"e40010ca7f3389080dc954dd2a7453583651a42ccd5894987bc2b302a9869f81"} Jul 22 02:04:55.638614 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:55.638588 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99" Jul 22 02:04:55.668023 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:55.667995 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15deff07-dad1-4b97-b1ec-fd653636c7c8-maas-api-service-ca\") pod \"15deff07-dad1-4b97-b1ec-fd653636c7c8\" (UID: \"15deff07-dad1-4b97-b1ec-fd653636c7c8\") " Jul 22 02:04:55.668424 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:55.668397 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15deff07-dad1-4b97-b1ec-fd653636c7c8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "15deff07-dad1-4b97-b1ec-fd653636c7c8" (UID: "15deff07-dad1-4b97-b1ec-fd653636c7c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:04:55.769781 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:55.769694 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/15deff07-dad1-4b97-b1ec-fd653636c7c8-maas-api-service-ca\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:04:56.509454 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:56.509420 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99" Jul 22 02:04:56.509454 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:56.509443 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99" event={"ID":"15deff07-dad1-4b97-b1ec-fd653636c7c8","Type":"ContainerDied","Data":"bf75e7eadcaa5ebd78d843951a9847d9ea4dff4731b9816953919a5004453a60"} Jul 22 02:04:56.509658 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:04:56.509515 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf75e7eadcaa5ebd78d843951a9847d9ea4dff4731b9816953919a5004453a60" Jul 22 02:05:07.523141 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:07.523107 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99"] Jul 22 02:05:07.526524 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:07.526499 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f2f0e842-sbv99"] Jul 22 02:05:07.625288 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:07.625252 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15deff07-dad1-4b97-b1ec-fd653636c7c8" path="/var/lib/kubelet/pods/15deff07-dad1-4b97-b1ec-fd653636c7c8/volumes" Jul 22 02:05:07.670041 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:07.670007 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2"] Jul 22 02:05:07.670302 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:07.670274 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" podUID="e93613b6-5403-4b04-8300-34ebe1c3b5b3" containerName="istio-proxy" containerID="cri-o://00d989f9224ca2e8d718a6155ddce4252bd903468df80fb1baca818e6295e113" gracePeriod=30 Jul 22 02:05:08.752502 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.752468 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc"] Jul 22 02:05:08.752894 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.752881 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15deff07-dad1-4b97-b1ec-fd653636c7c8" containerName="revoke-keys" Jul 22 02:05:08.752894 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.752895 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="15deff07-dad1-4b97-b1ec-fd653636c7c8" containerName="revoke-keys" Jul 22 02:05:08.752984 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.752975 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="15deff07-dad1-4b97-b1ec-fd653636c7c8" containerName="revoke-keys" Jul 22 02:05:08.756195 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.756174 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.758506 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.758486 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-0dd62ba0-gw-openshift-default-dockercfg-tpr7z\"" Jul 22 02:05:08.766777 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.766757 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc"] Jul 22 02:05:08.795136 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.795105 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttrmt\" (UniqueName: \"kubernetes.io/projected/582fa5c1-2464-46a2-ab08-6fe9add604c1-kube-api-access-ttrmt\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.795278 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.795157 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/582fa5c1-2464-46a2-ab08-6fe9add604c1-istiod-ca-cert\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.795278 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.795245 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-credential-socket\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.795432 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.795288 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-data\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.795432 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.795337 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-podinfo\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.795432 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.795406 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-workload-certs\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.795614 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.795441 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-envoy\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.795614 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.795469 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-token\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.795614 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.795499 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-workload-socket\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.896490 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.896451 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/582fa5c1-2464-46a2-ab08-6fe9add604c1-istiod-ca-cert\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.896686 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.896508 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-credential-socket\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.896686 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.896528 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-data\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.896686 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.896555 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-podinfo\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.896686 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.896578 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-workload-certs\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.896686 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.896599 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-envoy\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.896686 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.896615 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-token\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.896686 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.896634 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-workload-socket\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.897062 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.896700 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttrmt\" (UniqueName: \"kubernetes.io/projected/582fa5c1-2464-46a2-ab08-6fe9add604c1-kube-api-access-ttrmt\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.897116 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.897083 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-credential-socket\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.897158 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.897125 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-workload-certs\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.897158 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.897142 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-data\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.897254 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.897218 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/582fa5c1-2464-46a2-ab08-6fe9add604c1-istiod-ca-cert\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.897288 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.897245 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-workload-socket\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.899220 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.899195 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-envoy\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.899317 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.899299 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-podinfo\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.904967 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.904946 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-token\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:08.905106 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:08.905089 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttrmt\" (UniqueName: \"kubernetes.io/projected/582fa5c1-2464-46a2-ab08-6fe9add604c1-kube-api-access-ttrmt\") pod \"e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:09.067548 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:09.067460 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:09.208911 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:09.208885 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc"] Jul 22 02:05:09.210998 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:05:09.210964 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod582fa5c1_2464_46a2_ab08_6fe9add604c1.slice/crio-0648a0a78dded89c3846cceebc1fc140af6a5b2dbda4c38b0e78cf2f3ecbd6dd WatchSource:0}: Error finding container 0648a0a78dded89c3846cceebc1fc140af6a5b2dbda4c38b0e78cf2f3ecbd6dd: Status 404 returned error can't find the container with id 0648a0a78dded89c3846cceebc1fc140af6a5b2dbda4c38b0e78cf2f3ecbd6dd Jul 22 02:05:09.213005 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:09.212972 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:05:09.213107 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:09.213047 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:05:09.213107 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:09.213090 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:05:09.572255 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:09.572219 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" event={"ID":"582fa5c1-2464-46a2-ab08-6fe9add604c1","Type":"ContainerStarted","Data":"ace8b2d280799c05c79a6f5125b040930055e76bc1888d734e859b09c89f4b73"} Jul 22 02:05:09.572443 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:09.572275 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" event={"ID":"582fa5c1-2464-46a2-ab08-6fe9add604c1","Type":"ContainerStarted","Data":"0648a0a78dded89c3846cceebc1fc140af6a5b2dbda4c38b0e78cf2f3ecbd6dd"} Jul 22 02:05:09.593507 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:09.593458 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" podStartSLOduration=1.593440745 podStartE2EDuration="1.593440745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:05:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:05:09.592075006 +0000 UTC m=+1694.421323357" watchObservedRunningTime="2026-07-22 02:05:09.593440745 +0000 UTC m=+1694.422689093" Jul 22 02:05:10.067964 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:10.067866 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:10.072888 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:10.072864 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:10.576699 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:10.576672 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:10.577915 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:10.577894 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:12.926669 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:12.926645 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:05:13.037336 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.037259 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-envoy\") pod \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " Jul 22 02:05:13.037336 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.037309 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-credential-socket\") pod \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " Jul 22 02:05:13.037562 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.037370 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-token\") pod \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " Jul 22 02:05:13.037562 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.037408 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-data\") pod \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " Jul 22 02:05:13.037562 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.037444 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98m7z\" (UniqueName: \"kubernetes.io/projected/e93613b6-5403-4b04-8300-34ebe1c3b5b3-kube-api-access-98m7z\") pod \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " Jul 22 02:05:13.037562 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.037500 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-workload-socket\") pod \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " Jul 22 02:05:13.037765 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.037602 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-podinfo\") pod \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " Jul 22 02:05:13.037765 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.037667 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-workload-certs\") pod \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " Jul 22 02:05:13.037765 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.037666 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e93613b6-5403-4b04-8300-34ebe1c3b5b3" (UID: "e93613b6-5403-4b04-8300-34ebe1c3b5b3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:05:13.037765 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.037721 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istiod-ca-cert\") pod \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\" (UID: \"e93613b6-5403-4b04-8300-34ebe1c3b5b3\") " Jul 22 02:05:13.038026 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.037771 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e93613b6-5403-4b04-8300-34ebe1c3b5b3" (UID: "e93613b6-5403-4b04-8300-34ebe1c3b5b3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:05:13.038026 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.037969 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e93613b6-5403-4b04-8300-34ebe1c3b5b3" (UID: "e93613b6-5403-4b04-8300-34ebe1c3b5b3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:05:13.038135 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.038126 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-workload-certs\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:05:13.038193 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.038144 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-credential-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:05:13.038193 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.038158 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-workload-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:05:13.038288 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.038253 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e93613b6-5403-4b04-8300-34ebe1c3b5b3" (UID: "e93613b6-5403-4b04-8300-34ebe1c3b5b3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:05:13.038288 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.038278 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-data" (OuterVolumeSpecName: "istio-data") pod "e93613b6-5403-4b04-8300-34ebe1c3b5b3" (UID: "e93613b6-5403-4b04-8300-34ebe1c3b5b3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:05:13.039933 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.039906 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e93613b6-5403-4b04-8300-34ebe1c3b5b3" (UID: "e93613b6-5403-4b04-8300-34ebe1c3b5b3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:05:13.040056 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.039933 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e93613b6-5403-4b04-8300-34ebe1c3b5b3-kube-api-access-98m7z" (OuterVolumeSpecName: "kube-api-access-98m7z") pod "e93613b6-5403-4b04-8300-34ebe1c3b5b3" (UID: "e93613b6-5403-4b04-8300-34ebe1c3b5b3"). InnerVolumeSpecName "kube-api-access-98m7z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:05:13.040056 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.039968 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-token" (OuterVolumeSpecName: "istio-token") pod "e93613b6-5403-4b04-8300-34ebe1c3b5b3" (UID: "e93613b6-5403-4b04-8300-34ebe1c3b5b3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:05:13.040056 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.039973 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e93613b6-5403-4b04-8300-34ebe1c3b5b3" (UID: "e93613b6-5403-4b04-8300-34ebe1c3b5b3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:05:13.139237 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.139210 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-envoy\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:05:13.139237 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.139231 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-token\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:05:13.139237 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.139240 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-data\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:05:13.139443 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.139252 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98m7z\" (UniqueName: \"kubernetes.io/projected/e93613b6-5403-4b04-8300-34ebe1c3b5b3-kube-api-access-98m7z\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:05:13.139443 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.139260 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istio-podinfo\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:05:13.139443 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.139268 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e93613b6-5403-4b04-8300-34ebe1c3b5b3-istiod-ca-cert\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:05:13.600613 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.600574 2559 generic.go:356] "Generic (PLEG): container finished" podID="e93613b6-5403-4b04-8300-34ebe1c3b5b3" containerID="00d989f9224ca2e8d718a6155ddce4252bd903468df80fb1baca818e6295e113" exitCode=0 Jul 22 02:05:13.600792 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.600640 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" Jul 22 02:05:13.600792 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.600655 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" event={"ID":"e93613b6-5403-4b04-8300-34ebe1c3b5b3","Type":"ContainerDied","Data":"00d989f9224ca2e8d718a6155ddce4252bd903468df80fb1baca818e6295e113"} Jul 22 02:05:13.600792 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.600750 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2" event={"ID":"e93613b6-5403-4b04-8300-34ebe1c3b5b3","Type":"ContainerDied","Data":"8115edea4e7c96bb32808de62dd3a15c8ea5c4520b84738b939571973f977bd6"} Jul 22 02:05:13.600792 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.600778 2559 scope.go:117] "RemoveContainer" containerID="00d989f9224ca2e8d718a6155ddce4252bd903468df80fb1baca818e6295e113" Jul 22 02:05:13.610300 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.610282 2559 scope.go:117] "RemoveContainer" containerID="b5cdc1d6aff7e0ea42c90b43bfd922c26408b240af52ab540d6fb91a76a4221f" Jul 22 02:05:13.618427 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.618410 2559 scope.go:117] "RemoveContainer" containerID="00d989f9224ca2e8d718a6155ddce4252bd903468df80fb1baca818e6295e113" Jul 22 02:05:13.618660 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:05:13.618642 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00d989f9224ca2e8d718a6155ddce4252bd903468df80fb1baca818e6295e113\": container with ID starting with 00d989f9224ca2e8d718a6155ddce4252bd903468df80fb1baca818e6295e113 not found: ID does not exist" containerID="00d989f9224ca2e8d718a6155ddce4252bd903468df80fb1baca818e6295e113" Jul 22 02:05:13.618713 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.618667 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00d989f9224ca2e8d718a6155ddce4252bd903468df80fb1baca818e6295e113"} err="failed to get container status \"00d989f9224ca2e8d718a6155ddce4252bd903468df80fb1baca818e6295e113\": rpc error: code = NotFound desc = could not find container \"00d989f9224ca2e8d718a6155ddce4252bd903468df80fb1baca818e6295e113\": container with ID starting with 00d989f9224ca2e8d718a6155ddce4252bd903468df80fb1baca818e6295e113 not found: ID does not exist" Jul 22 02:05:13.618713 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.618683 2559 scope.go:117] "RemoveContainer" containerID="b5cdc1d6aff7e0ea42c90b43bfd922c26408b240af52ab540d6fb91a76a4221f" Jul 22 02:05:13.618899 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:05:13.618880 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5cdc1d6aff7e0ea42c90b43bfd922c26408b240af52ab540d6fb91a76a4221f\": container with ID starting with b5cdc1d6aff7e0ea42c90b43bfd922c26408b240af52ab540d6fb91a76a4221f not found: ID does not exist" containerID="b5cdc1d6aff7e0ea42c90b43bfd922c26408b240af52ab540d6fb91a76a4221f" Jul 22 02:05:13.618947 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.618908 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5cdc1d6aff7e0ea42c90b43bfd922c26408b240af52ab540d6fb91a76a4221f"} err="failed to get container status \"b5cdc1d6aff7e0ea42c90b43bfd922c26408b240af52ab540d6fb91a76a4221f\": rpc error: code = NotFound desc = could not find container \"b5cdc1d6aff7e0ea42c90b43bfd922c26408b240af52ab540d6fb91a76a4221f\": container with ID starting with b5cdc1d6aff7e0ea42c90b43bfd922c26408b240af52ab540d6fb91a76a4221f not found: ID does not exist" Jul 22 02:05:13.627267 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.627246 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2"] Jul 22 02:05:13.628449 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:13.628431 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f2f0e842-openshift-default-85b97fbb77-5z8v2"] Jul 22 02:05:15.625195 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:15.625163 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e93613b6-5403-4b04-8300-34ebe1c3b5b3" path="/var/lib/kubelet/pods/e93613b6-5403-4b04-8300-34ebe1c3b5b3/volumes" Jul 22 02:05:26.157305 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:26.157248 2559 patch_prober.go:28] interesting pod/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": read tcp 10.132.0.2:40856->10.132.0.37:15021: read: connection reset by peer" start-of-body= Jul 22 02:05:26.157745 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:26.157323 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" podUID="582fa5c1-2464-46a2-ab08-6fe9add604c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": read tcp 10.132.0.2:40856->10.132.0.37:15021: read: connection reset by peer" Jul 22 02:05:26.687123 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:26.687089 2559 generic.go:356] "Generic (PLEG): container finished" podID="582fa5c1-2464-46a2-ab08-6fe9add604c1" containerID="ace8b2d280799c05c79a6f5125b040930055e76bc1888d734e859b09c89f4b73" exitCode=0 Jul 22 02:05:26.687291 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:26.687160 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" event={"ID":"582fa5c1-2464-46a2-ab08-6fe9add604c1","Type":"ContainerDied","Data":"ace8b2d280799c05c79a6f5125b040930055e76bc1888d734e859b09c89f4b73"} Jul 22 02:05:26.687609 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:26.687590 2559 scope.go:117] "RemoveContainer" containerID="ace8b2d280799c05c79a6f5125b040930055e76bc1888d734e859b09c89f4b73" Jul 22 02:05:26.688583 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:26.688555 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:05:26.688653 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:26.688620 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:05:26.688690 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:26.688650 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:05:27.693109 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:27.693076 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" event={"ID":"582fa5c1-2464-46a2-ab08-6fe9add604c1","Type":"ContainerStarted","Data":"669d8539596231ecc2b1eefae11a2c8315b02146105f1165100935764fa09be3"} Jul 22 02:05:28.068185 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:28.068098 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:28.068368 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:28.068300 2559 patch_prober.go:28] interesting pod/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 22 02:05:28.068429 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:28.068406 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" podUID="582fa5c1-2464-46a2-ab08-6fe9add604c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 22 02:05:29.067980 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:29.067945 2559 patch_prober.go:28] interesting pod/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 22 02:05:29.068409 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:29.068018 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" podUID="582fa5c1-2464-46a2-ab08-6fe9add604c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 22 02:05:30.068206 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:30.068172 2559 patch_prober.go:28] interesting pod/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 22 02:05:30.068627 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:30.068229 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" podUID="582fa5c1-2464-46a2-ab08-6fe9add604c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 22 02:05:31.067802 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:31.067760 2559 patch_prober.go:28] interesting pod/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 22 02:05:31.068005 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:31.067820 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" podUID="582fa5c1-2464-46a2-ab08-6fe9add604c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 22 02:05:32.071045 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:32.071014 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:32.071468 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:32.071383 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:32.072067 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:32.072048 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:36.150895 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:36.150861 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0dd62ba0-9d85t"] Jul 22 02:05:36.151258 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:36.151237 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e93613b6-5403-4b04-8300-34ebe1c3b5b3" containerName="istio-proxy" Jul 22 02:05:36.151258 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:36.151250 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e93613b6-5403-4b04-8300-34ebe1c3b5b3" containerName="istio-proxy" Jul 22 02:05:36.151329 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:36.151265 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e93613b6-5403-4b04-8300-34ebe1c3b5b3" containerName="istio-proxy" Jul 22 02:05:36.151329 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:36.151273 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e93613b6-5403-4b04-8300-34ebe1c3b5b3" containerName="istio-proxy" Jul 22 02:05:36.151329 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:36.151322 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e93613b6-5403-4b04-8300-34ebe1c3b5b3" containerName="istio-proxy" Jul 22 02:05:36.151458 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:36.151333 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e93613b6-5403-4b04-8300-34ebe1c3b5b3" containerName="istio-proxy" Jul 22 02:05:36.154638 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:36.154621 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0dd62ba0-9d85t" Jul 22 02:05:36.156876 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:36.156846 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cqdw8\"" Jul 22 02:05:36.157000 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:36.156950 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 02:05:36.160909 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:36.160888 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0dd62ba0-9d85t"] Jul 22 02:05:36.262526 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:36.262492 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eee82623-888d-4b6e-8d1c-afb9a741781c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0dd62ba0-9d85t\" (UID: \"eee82623-888d-4b6e-8d1c-afb9a741781c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0dd62ba0-9d85t" Jul 22 02:05:36.363917 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:36.363886 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eee82623-888d-4b6e-8d1c-afb9a741781c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0dd62ba0-9d85t\" (UID: \"eee82623-888d-4b6e-8d1c-afb9a741781c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0dd62ba0-9d85t" Jul 22 02:05:36.364498 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:36.364480 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eee82623-888d-4b6e-8d1c-afb9a741781c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0dd62ba0-9d85t\" (UID: \"eee82623-888d-4b6e-8d1c-afb9a741781c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0dd62ba0-9d85t" Jul 22 02:05:36.467003 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:36.466949 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0dd62ba0-9d85t" Jul 22 02:05:36.596215 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:36.596189 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0dd62ba0-9d85t"] Jul 22 02:05:36.598753 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:05:36.598722 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeee82623_888d_4b6e_8d1c_afb9a741781c.slice/crio-fe797d91f0074ba0a438714b29a6eaa84cc5de43122bffc308f87b7f567efc31 WatchSource:0}: Error finding container fe797d91f0074ba0a438714b29a6eaa84cc5de43122bffc308f87b7f567efc31: Status 404 returned error can't find the container with id fe797d91f0074ba0a438714b29a6eaa84cc5de43122bffc308f87b7f567efc31 Jul 22 02:05:36.736765 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:36.736693 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0dd62ba0-9d85t" event={"ID":"eee82623-888d-4b6e-8d1c-afb9a741781c","Type":"ContainerStarted","Data":"fe797d91f0074ba0a438714b29a6eaa84cc5de43122bffc308f87b7f567efc31"} Jul 22 02:05:37.742488 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:37.742413 2559 generic.go:356] "Generic (PLEG): container finished" podID="eee82623-888d-4b6e-8d1c-afb9a741781c" containerID="203b79e456986172134a4c96ebd245fc6262c454fba35a035d1eeb135a0dea0e" exitCode=0 Jul 22 02:05:37.742488 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:37.742468 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0dd62ba0-9d85t" event={"ID":"eee82623-888d-4b6e-8d1c-afb9a741781c","Type":"ContainerDied","Data":"203b79e456986172134a4c96ebd245fc6262c454fba35a035d1eeb135a0dea0e"} Jul 22 02:05:38.883674 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:38.883650 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0dd62ba0-9d85t" Jul 22 02:05:38.988691 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:38.988663 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eee82623-888d-4b6e-8d1c-afb9a741781c-maas-api-service-ca\") pod \"eee82623-888d-4b6e-8d1c-afb9a741781c\" (UID: \"eee82623-888d-4b6e-8d1c-afb9a741781c\") " Jul 22 02:05:38.989028 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:38.989008 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eee82623-888d-4b6e-8d1c-afb9a741781c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eee82623-888d-4b6e-8d1c-afb9a741781c" (UID: "eee82623-888d-4b6e-8d1c-afb9a741781c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:05:39.090175 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:39.090151 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eee82623-888d-4b6e-8d1c-afb9a741781c-maas-api-service-ca\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:05:39.752588 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:39.752556 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0dd62ba0-9d85t" Jul 22 02:05:39.752588 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:39.752566 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0dd62ba0-9d85t" event={"ID":"eee82623-888d-4b6e-8d1c-afb9a741781c","Type":"ContainerDied","Data":"fe797d91f0074ba0a438714b29a6eaa84cc5de43122bffc308f87b7f567efc31"} Jul 22 02:05:39.752781 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:39.752614 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe797d91f0074ba0a438714b29a6eaa84cc5de43122bffc308f87b7f567efc31" Jul 22 02:05:51.237022 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:51.236981 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0dd62ba0-9d85t"] Jul 22 02:05:51.239232 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:51.239208 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0dd62ba0-9d85t"] Jul 22 02:05:51.387610 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:51.387574 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc"] Jul 22 02:05:51.387821 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:51.387783 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" podUID="582fa5c1-2464-46a2-ab08-6fe9add604c1" containerName="istio-proxy" containerID="cri-o://669d8539596231ecc2b1eefae11a2c8315b02146105f1165100935764fa09be3" gracePeriod=30 Jul 22 02:05:51.625311 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:51.625268 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eee82623-888d-4b6e-8d1c-afb9a741781c" path="/var/lib/kubelet/pods/eee82623-888d-4b6e-8d1c-afb9a741781c/volumes" Jul 22 02:05:52.207725 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.207692 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf"] Jul 22 02:05:52.208087 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.208074 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eee82623-888d-4b6e-8d1c-afb9a741781c" containerName="revoke-keys" Jul 22 02:05:52.208142 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.208089 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="eee82623-888d-4b6e-8d1c-afb9a741781c" containerName="revoke-keys" Jul 22 02:05:52.208184 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.208163 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="eee82623-888d-4b6e-8d1c-afb9a741781c" containerName="revoke-keys" Jul 22 02:05:52.211755 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.211737 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.213976 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.213950 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8662596b-openshift-default-dockercfg-8rllr\"" Jul 22 02:05:52.223034 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.223005 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf"] Jul 22 02:05:52.300485 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.300446 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-workload-certs\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.300917 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.300500 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-token\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.300917 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.300569 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-data\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.300917 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.300618 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/184f39b1-faf3-4dcb-8597-ca709e452a5f-istiod-ca-cert\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.300917 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.300674 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-credential-socket\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.300917 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.300709 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-workload-socket\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.300917 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.300779 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-envoy\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.300917 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.300868 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-podinfo\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.300917 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.300903 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmqn7\" (UniqueName: \"kubernetes.io/projected/184f39b1-faf3-4dcb-8597-ca709e452a5f-kube-api-access-kmqn7\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.401786 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.401745 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-credential-socket\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.401958 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.401795 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-workload-socket\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.401958 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.401823 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-envoy\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.401958 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.401879 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-podinfo\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.401958 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.401901 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmqn7\" (UniqueName: \"kubernetes.io/projected/184f39b1-faf3-4dcb-8597-ca709e452a5f-kube-api-access-kmqn7\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.402195 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.401960 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-workload-certs\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.402195 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.401985 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-token\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.402195 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.402022 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-data\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.402195 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.402068 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/184f39b1-faf3-4dcb-8597-ca709e452a5f-istiod-ca-cert\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.402434 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.402239 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-workload-socket\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.402434 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.402292 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-credential-socket\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.402545 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.402428 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-workload-certs\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.402545 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.402485 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-data\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.402688 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.402671 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/184f39b1-faf3-4dcb-8597-ca709e452a5f-istiod-ca-cert\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.404383 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.404364 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-envoy\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.404606 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.404589 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-podinfo\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.412380 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.412203 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-token\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.412380 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.412368 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmqn7\" (UniqueName: \"kubernetes.io/projected/184f39b1-faf3-4dcb-8597-ca709e452a5f-kube-api-access-kmqn7\") pod \"e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.522869 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.522778 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:52.665304 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.665278 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf"] Jul 22 02:05:52.667587 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:05:52.667562 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod184f39b1_faf3_4dcb_8597_ca709e452a5f.slice/crio-cf593b9223234d7408db79ac3ac0da1eff15c06e073f90242933bc0c75caae03 WatchSource:0}: Error finding container cf593b9223234d7408db79ac3ac0da1eff15c06e073f90242933bc0c75caae03: Status 404 returned error can't find the container with id cf593b9223234d7408db79ac3ac0da1eff15c06e073f90242933bc0c75caae03 Jul 22 02:05:52.669641 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.669608 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:05:52.669724 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.669673 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:05:52.669724 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.669699 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:05:52.819610 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.819510 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" event={"ID":"184f39b1-faf3-4dcb-8597-ca709e452a5f","Type":"ContainerStarted","Data":"fe3f2aba31143d8b7821adaec90ca649fd716d3b30907314bf4dfd9499d9d272"} Jul 22 02:05:52.819610 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.819580 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" event={"ID":"184f39b1-faf3-4dcb-8597-ca709e452a5f","Type":"ContainerStarted","Data":"cf593b9223234d7408db79ac3ac0da1eff15c06e073f90242933bc0c75caae03"} Jul 22 02:05:52.840597 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:52.840539 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" podStartSLOduration=0.840524775 podStartE2EDuration="840.524775ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:05:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:05:52.839366402 +0000 UTC m=+1737.668614750" watchObservedRunningTime="2026-07-22 02:05:52.840524775 +0000 UTC m=+1737.669773193" Jul 22 02:05:53.523994 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:53.523953 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:53.528980 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:53.528954 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:53.824338 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:53.824307 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:53.825295 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:53.825267 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:05:56.642770 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.642745 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:56.745825 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.745721 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-token\") pod \"582fa5c1-2464-46a2-ab08-6fe9add604c1\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " Jul 22 02:05:56.745825 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.745790 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-podinfo\") pod \"582fa5c1-2464-46a2-ab08-6fe9add604c1\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " Jul 22 02:05:56.745825 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.745817 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-workload-certs\") pod \"582fa5c1-2464-46a2-ab08-6fe9add604c1\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " Jul 22 02:05:56.746120 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.745854 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-credential-socket\") pod \"582fa5c1-2464-46a2-ab08-6fe9add604c1\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " Jul 22 02:05:56.746120 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.745884 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-workload-socket\") pod \"582fa5c1-2464-46a2-ab08-6fe9add604c1\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " Jul 22 02:05:56.746120 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.745919 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-data\") pod \"582fa5c1-2464-46a2-ab08-6fe9add604c1\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " Jul 22 02:05:56.746120 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.745944 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-envoy\") pod \"582fa5c1-2464-46a2-ab08-6fe9add604c1\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " Jul 22 02:05:56.746120 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.745975 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttrmt\" (UniqueName: \"kubernetes.io/projected/582fa5c1-2464-46a2-ab08-6fe9add604c1-kube-api-access-ttrmt\") pod \"582fa5c1-2464-46a2-ab08-6fe9add604c1\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " Jul 22 02:05:56.746120 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.746025 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/582fa5c1-2464-46a2-ab08-6fe9add604c1-istiod-ca-cert\") pod \"582fa5c1-2464-46a2-ab08-6fe9add604c1\" (UID: \"582fa5c1-2464-46a2-ab08-6fe9add604c1\") " Jul 22 02:05:56.746445 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.746166 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "582fa5c1-2464-46a2-ab08-6fe9add604c1" (UID: "582fa5c1-2464-46a2-ab08-6fe9add604c1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:05:56.746445 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.746173 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "582fa5c1-2464-46a2-ab08-6fe9add604c1" (UID: "582fa5c1-2464-46a2-ab08-6fe9add604c1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:05:56.746445 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.746398 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "582fa5c1-2464-46a2-ab08-6fe9add604c1" (UID: "582fa5c1-2464-46a2-ab08-6fe9add604c1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:05:56.746445 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.746414 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-workload-certs\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:05:56.746445 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.746434 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-credential-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:05:56.746648 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.746529 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/582fa5c1-2464-46a2-ab08-6fe9add604c1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "582fa5c1-2464-46a2-ab08-6fe9add604c1" (UID: "582fa5c1-2464-46a2-ab08-6fe9add604c1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:05:56.747283 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.747255 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-data" (OuterVolumeSpecName: "istio-data") pod "582fa5c1-2464-46a2-ab08-6fe9add604c1" (UID: "582fa5c1-2464-46a2-ab08-6fe9add604c1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:05:56.748016 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.747986 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "582fa5c1-2464-46a2-ab08-6fe9add604c1" (UID: "582fa5c1-2464-46a2-ab08-6fe9add604c1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:05:56.748152 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.748133 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-token" (OuterVolumeSpecName: "istio-token") pod "582fa5c1-2464-46a2-ab08-6fe9add604c1" (UID: "582fa5c1-2464-46a2-ab08-6fe9add604c1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:05:56.748247 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.748229 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/582fa5c1-2464-46a2-ab08-6fe9add604c1-kube-api-access-ttrmt" (OuterVolumeSpecName: "kube-api-access-ttrmt") pod "582fa5c1-2464-46a2-ab08-6fe9add604c1" (UID: "582fa5c1-2464-46a2-ab08-6fe9add604c1"). InnerVolumeSpecName "kube-api-access-ttrmt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:05:56.748877 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.748853 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "582fa5c1-2464-46a2-ab08-6fe9add604c1" (UID: "582fa5c1-2464-46a2-ab08-6fe9add604c1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:05:56.840341 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.840304 2559 generic.go:356] "Generic (PLEG): container finished" podID="582fa5c1-2464-46a2-ab08-6fe9add604c1" containerID="669d8539596231ecc2b1eefae11a2c8315b02146105f1165100935764fa09be3" exitCode=0 Jul 22 02:05:56.840536 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.840375 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" Jul 22 02:05:56.840536 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.840383 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" event={"ID":"582fa5c1-2464-46a2-ab08-6fe9add604c1","Type":"ContainerDied","Data":"669d8539596231ecc2b1eefae11a2c8315b02146105f1165100935764fa09be3"} Jul 22 02:05:56.840536 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.840471 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc" event={"ID":"582fa5c1-2464-46a2-ab08-6fe9add604c1","Type":"ContainerDied","Data":"0648a0a78dded89c3846cceebc1fc140af6a5b2dbda4c38b0e78cf2f3ecbd6dd"} Jul 22 02:05:56.840536 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.840497 2559 scope.go:117] "RemoveContainer" containerID="669d8539596231ecc2b1eefae11a2c8315b02146105f1165100935764fa09be3" Jul 22 02:05:56.847201 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.847175 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-token\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:05:56.847307 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.847204 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-podinfo\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:05:56.847307 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.847217 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-workload-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:05:56.847307 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.847229 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-data\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:05:56.847307 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.847241 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/582fa5c1-2464-46a2-ab08-6fe9add604c1-istio-envoy\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:05:56.847307 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.847252 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttrmt\" (UniqueName: \"kubernetes.io/projected/582fa5c1-2464-46a2-ab08-6fe9add604c1-kube-api-access-ttrmt\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:05:56.847307 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.847265 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/582fa5c1-2464-46a2-ab08-6fe9add604c1-istiod-ca-cert\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:05:56.849801 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.849785 2559 scope.go:117] "RemoveContainer" containerID="ace8b2d280799c05c79a6f5125b040930055e76bc1888d734e859b09c89f4b73" Jul 22 02:05:56.857799 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.857778 2559 scope.go:117] "RemoveContainer" containerID="669d8539596231ecc2b1eefae11a2c8315b02146105f1165100935764fa09be3" Jul 22 02:05:56.858073 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:05:56.858052 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"669d8539596231ecc2b1eefae11a2c8315b02146105f1165100935764fa09be3\": container with ID starting with 669d8539596231ecc2b1eefae11a2c8315b02146105f1165100935764fa09be3 not found: ID does not exist" containerID="669d8539596231ecc2b1eefae11a2c8315b02146105f1165100935764fa09be3" Jul 22 02:05:56.858129 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.858082 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"669d8539596231ecc2b1eefae11a2c8315b02146105f1165100935764fa09be3"} err="failed to get container status \"669d8539596231ecc2b1eefae11a2c8315b02146105f1165100935764fa09be3\": rpc error: code = NotFound desc = could not find container \"669d8539596231ecc2b1eefae11a2c8315b02146105f1165100935764fa09be3\": container with ID starting with 669d8539596231ecc2b1eefae11a2c8315b02146105f1165100935764fa09be3 not found: ID does not exist" Jul 22 02:05:56.858129 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.858101 2559 scope.go:117] "RemoveContainer" containerID="ace8b2d280799c05c79a6f5125b040930055e76bc1888d734e859b09c89f4b73" Jul 22 02:05:56.858374 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:05:56.858321 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ace8b2d280799c05c79a6f5125b040930055e76bc1888d734e859b09c89f4b73\": container with ID starting with ace8b2d280799c05c79a6f5125b040930055e76bc1888d734e859b09c89f4b73 not found: ID does not exist" containerID="ace8b2d280799c05c79a6f5125b040930055e76bc1888d734e859b09c89f4b73" Jul 22 02:05:56.858464 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.858381 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ace8b2d280799c05c79a6f5125b040930055e76bc1888d734e859b09c89f4b73"} err="failed to get container status \"ace8b2d280799c05c79a6f5125b040930055e76bc1888d734e859b09c89f4b73\": rpc error: code = NotFound desc = could not find container \"ace8b2d280799c05c79a6f5125b040930055e76bc1888d734e859b09c89f4b73\": container with ID starting with ace8b2d280799c05c79a6f5125b040930055e76bc1888d734e859b09c89f4b73 not found: ID does not exist" Jul 22 02:05:56.862819 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.862794 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc"] Jul 22 02:05:56.866931 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:56.866908 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-0dd62ba0-gw-openshift-default-8b7985b7f-bt9cc"] Jul 22 02:05:57.626653 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:05:57.626623 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="582fa5c1-2464-46a2-ab08-6fe9add604c1" path="/var/lib/kubelet/pods/582fa5c1-2464-46a2-ab08-6fe9add604c1/volumes" Jul 22 02:06:09.612564 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:09.611046 2559 patch_prober.go:28] interesting pod/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": read tcp 10.132.0.2:35700->10.132.0.39:15021: read: connection reset by peer" start-of-body= Jul 22 02:06:09.612564 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:09.611140 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" podUID="184f39b1-faf3-4dcb-8597-ca709e452a5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": read tcp 10.132.0.2:35700->10.132.0.39:15021: read: connection reset by peer" Jul 22 02:06:09.910419 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:09.910321 2559 generic.go:356] "Generic (PLEG): container finished" podID="184f39b1-faf3-4dcb-8597-ca709e452a5f" containerID="fe3f2aba31143d8b7821adaec90ca649fd716d3b30907314bf4dfd9499d9d272" exitCode=0 Jul 22 02:06:09.910419 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:09.910388 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" event={"ID":"184f39b1-faf3-4dcb-8597-ca709e452a5f","Type":"ContainerDied","Data":"fe3f2aba31143d8b7821adaec90ca649fd716d3b30907314bf4dfd9499d9d272"} Jul 22 02:06:09.910785 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:09.910772 2559 scope.go:117] "RemoveContainer" containerID="fe3f2aba31143d8b7821adaec90ca649fd716d3b30907314bf4dfd9499d9d272" Jul 22 02:06:09.911850 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:09.911813 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:06:09.911957 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:09.911903 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:06:09.911957 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:09.911946 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:06:10.917912 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:10.917882 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" event={"ID":"184f39b1-faf3-4dcb-8597-ca709e452a5f","Type":"ContainerStarted","Data":"7d4663d6cb13752df23fd96ae53a7937793c474adefbcedef3a3622220117803"} Jul 22 02:06:11.523169 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:11.523131 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:06:11.523430 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:11.523394 2559 patch_prober.go:28] interesting pod/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 22 02:06:11.523485 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:11.523456 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" podUID="184f39b1-faf3-4dcb-8597-ca709e452a5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 22 02:06:12.523658 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:12.523622 2559 patch_prober.go:28] interesting pod/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 22 02:06:12.524038 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:12.523677 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" podUID="184f39b1-faf3-4dcb-8597-ca709e452a5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 22 02:06:13.523539 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:13.523498 2559 patch_prober.go:28] interesting pod/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 22 02:06:13.523726 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:13.523568 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" podUID="184f39b1-faf3-4dcb-8597-ca709e452a5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 22 02:06:14.523925 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:14.523884 2559 patch_prober.go:28] interesting pod/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 22 02:06:14.524376 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:14.523942 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" podUID="184f39b1-faf3-4dcb-8597-ca709e452a5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 22 02:06:15.526656 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:15.526623 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:06:15.526993 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:15.526885 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:06:15.527596 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:15.527575 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:06:36.502614 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:36.502581 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf"] Jul 22 02:06:36.503048 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:36.502781 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" podUID="184f39b1-faf3-4dcb-8597-ca709e452a5f" containerName="istio-proxy" containerID="cri-o://7d4663d6cb13752df23fd96ae53a7937793c474adefbcedef3a3622220117803" gracePeriod=30 Jul 22 02:06:41.761446 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.761377 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:06:41.788535 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.788506 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-credential-socket\") pod \"184f39b1-faf3-4dcb-8597-ca709e452a5f\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " Jul 22 02:06:41.788692 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.788563 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-workload-certs\") pod \"184f39b1-faf3-4dcb-8597-ca709e452a5f\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " Jul 22 02:06:41.788692 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.788588 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmqn7\" (UniqueName: \"kubernetes.io/projected/184f39b1-faf3-4dcb-8597-ca709e452a5f-kube-api-access-kmqn7\") pod \"184f39b1-faf3-4dcb-8597-ca709e452a5f\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " Jul 22 02:06:41.788692 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.788610 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-workload-socket\") pod \"184f39b1-faf3-4dcb-8597-ca709e452a5f\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " Jul 22 02:06:41.788692 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.788629 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-token\") pod \"184f39b1-faf3-4dcb-8597-ca709e452a5f\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " Jul 22 02:06:41.788916 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.788732 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-envoy\") pod \"184f39b1-faf3-4dcb-8597-ca709e452a5f\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " Jul 22 02:06:41.788916 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.788803 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-data\") pod \"184f39b1-faf3-4dcb-8597-ca709e452a5f\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " Jul 22 02:06:41.788916 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.788803 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "184f39b1-faf3-4dcb-8597-ca709e452a5f" (UID: "184f39b1-faf3-4dcb-8597-ca709e452a5f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:06:41.788916 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.788878 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/184f39b1-faf3-4dcb-8597-ca709e452a5f-istiod-ca-cert\") pod \"184f39b1-faf3-4dcb-8597-ca709e452a5f\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " Jul 22 02:06:41.788916 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.788893 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "184f39b1-faf3-4dcb-8597-ca709e452a5f" (UID: "184f39b1-faf3-4dcb-8597-ca709e452a5f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:06:41.788916 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.788916 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-podinfo\") pod \"184f39b1-faf3-4dcb-8597-ca709e452a5f\" (UID: \"184f39b1-faf3-4dcb-8597-ca709e452a5f\") " Jul 22 02:06:41.789236 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.788911 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "184f39b1-faf3-4dcb-8597-ca709e452a5f" (UID: "184f39b1-faf3-4dcb-8597-ca709e452a5f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:06:41.789294 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.789240 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-workload-certs\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:06:41.789294 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.789259 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-workload-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:06:41.789294 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.789273 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-credential-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:06:41.789294 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.789254 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/184f39b1-faf3-4dcb-8597-ca709e452a5f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "184f39b1-faf3-4dcb-8597-ca709e452a5f" (UID: "184f39b1-faf3-4dcb-8597-ca709e452a5f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:06:41.789803 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.789773 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-data" (OuterVolumeSpecName: "istio-data") pod "184f39b1-faf3-4dcb-8597-ca709e452a5f" (UID: "184f39b1-faf3-4dcb-8597-ca709e452a5f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:06:41.790839 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.790813 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-token" (OuterVolumeSpecName: "istio-token") pod "184f39b1-faf3-4dcb-8597-ca709e452a5f" (UID: "184f39b1-faf3-4dcb-8597-ca709e452a5f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:06:41.791048 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.791026 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/184f39b1-faf3-4dcb-8597-ca709e452a5f-kube-api-access-kmqn7" (OuterVolumeSpecName: "kube-api-access-kmqn7") pod "184f39b1-faf3-4dcb-8597-ca709e452a5f" (UID: "184f39b1-faf3-4dcb-8597-ca709e452a5f"). InnerVolumeSpecName "kube-api-access-kmqn7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:06:41.791205 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.791172 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "184f39b1-faf3-4dcb-8597-ca709e452a5f" (UID: "184f39b1-faf3-4dcb-8597-ca709e452a5f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:06:41.791295 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.791219 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "184f39b1-faf3-4dcb-8597-ca709e452a5f" (UID: "184f39b1-faf3-4dcb-8597-ca709e452a5f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:06:41.890496 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.890457 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-envoy\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:06:41.890496 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.890489 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-data\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:06:41.890496 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.890501 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/184f39b1-faf3-4dcb-8597-ca709e452a5f-istiod-ca-cert\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:06:41.890744 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.890512 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-podinfo\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:06:41.890744 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.890524 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmqn7\" (UniqueName: \"kubernetes.io/projected/184f39b1-faf3-4dcb-8597-ca709e452a5f-kube-api-access-kmqn7\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:06:41.890744 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:41.890536 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/184f39b1-faf3-4dcb-8597-ca709e452a5f-istio-token\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:06:42.075166 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:42.075134 2559 generic.go:356] "Generic (PLEG): container finished" podID="184f39b1-faf3-4dcb-8597-ca709e452a5f" containerID="7d4663d6cb13752df23fd96ae53a7937793c474adefbcedef3a3622220117803" exitCode=0 Jul 22 02:06:42.075340 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:42.075188 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" Jul 22 02:06:42.075340 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:42.075221 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" event={"ID":"184f39b1-faf3-4dcb-8597-ca709e452a5f","Type":"ContainerDied","Data":"7d4663d6cb13752df23fd96ae53a7937793c474adefbcedef3a3622220117803"} Jul 22 02:06:42.075340 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:42.075271 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf" event={"ID":"184f39b1-faf3-4dcb-8597-ca709e452a5f","Type":"ContainerDied","Data":"cf593b9223234d7408db79ac3ac0da1eff15c06e073f90242933bc0c75caae03"} Jul 22 02:06:42.075340 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:42.075295 2559 scope.go:117] "RemoveContainer" containerID="7d4663d6cb13752df23fd96ae53a7937793c474adefbcedef3a3622220117803" Jul 22 02:06:42.084557 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:42.084538 2559 scope.go:117] "RemoveContainer" containerID="fe3f2aba31143d8b7821adaec90ca649fd716d3b30907314bf4dfd9499d9d272" Jul 22 02:06:42.091939 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:42.091921 2559 scope.go:117] "RemoveContainer" containerID="7d4663d6cb13752df23fd96ae53a7937793c474adefbcedef3a3622220117803" Jul 22 02:06:42.092199 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:06:42.092175 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d4663d6cb13752df23fd96ae53a7937793c474adefbcedef3a3622220117803\": container with ID starting with 7d4663d6cb13752df23fd96ae53a7937793c474adefbcedef3a3622220117803 not found: ID does not exist" containerID="7d4663d6cb13752df23fd96ae53a7937793c474adefbcedef3a3622220117803" Jul 22 02:06:42.092272 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:42.092207 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d4663d6cb13752df23fd96ae53a7937793c474adefbcedef3a3622220117803"} err="failed to get container status \"7d4663d6cb13752df23fd96ae53a7937793c474adefbcedef3a3622220117803\": rpc error: code = NotFound desc = could not find container \"7d4663d6cb13752df23fd96ae53a7937793c474adefbcedef3a3622220117803\": container with ID starting with 7d4663d6cb13752df23fd96ae53a7937793c474adefbcedef3a3622220117803 not found: ID does not exist" Jul 22 02:06:42.092272 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:42.092224 2559 scope.go:117] "RemoveContainer" containerID="fe3f2aba31143d8b7821adaec90ca649fd716d3b30907314bf4dfd9499d9d272" Jul 22 02:06:42.092479 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:06:42.092462 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe3f2aba31143d8b7821adaec90ca649fd716d3b30907314bf4dfd9499d9d272\": container with ID starting with fe3f2aba31143d8b7821adaec90ca649fd716d3b30907314bf4dfd9499d9d272 not found: ID does not exist" containerID="fe3f2aba31143d8b7821adaec90ca649fd716d3b30907314bf4dfd9499d9d272" Jul 22 02:06:42.092533 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:42.092482 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe3f2aba31143d8b7821adaec90ca649fd716d3b30907314bf4dfd9499d9d272"} err="failed to get container status \"fe3f2aba31143d8b7821adaec90ca649fd716d3b30907314bf4dfd9499d9d272\": rpc error: code = NotFound desc = could not find container \"fe3f2aba31143d8b7821adaec90ca649fd716d3b30907314bf4dfd9499d9d272\": container with ID starting with fe3f2aba31143d8b7821adaec90ca649fd716d3b30907314bf4dfd9499d9d272 not found: ID does not exist" Jul 22 02:06:42.101833 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:42.100693 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf"] Jul 22 02:06:42.105017 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:42.104980 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8662596b-openshift-default-7cbd97dbdc-7p8lf"] Jul 22 02:06:43.625458 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:43.625421 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="184f39b1-faf3-4dcb-8597-ca709e452a5f" path="/var/lib/kubelet/pods/184f39b1-faf3-4dcb-8597-ca709e452a5f/volumes" Jul 22 02:06:46.294754 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.294722 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx"] Jul 22 02:06:46.295262 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.295243 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="184f39b1-faf3-4dcb-8597-ca709e452a5f" containerName="istio-proxy" Jul 22 02:06:46.295358 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.295265 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="184f39b1-faf3-4dcb-8597-ca709e452a5f" containerName="istio-proxy" Jul 22 02:06:46.295358 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.295278 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="184f39b1-faf3-4dcb-8597-ca709e452a5f" containerName="istio-proxy" Jul 22 02:06:46.295358 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.295287 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="184f39b1-faf3-4dcb-8597-ca709e452a5f" containerName="istio-proxy" Jul 22 02:06:46.295358 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.295322 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="582fa5c1-2464-46a2-ab08-6fe9add604c1" containerName="istio-proxy" Jul 22 02:06:46.295358 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.295331 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="582fa5c1-2464-46a2-ab08-6fe9add604c1" containerName="istio-proxy" Jul 22 02:06:46.295617 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.295361 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="582fa5c1-2464-46a2-ab08-6fe9add604c1" containerName="istio-proxy" Jul 22 02:06:46.295617 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.295370 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="582fa5c1-2464-46a2-ab08-6fe9add604c1" containerName="istio-proxy" Jul 22 02:06:46.295617 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.295480 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="582fa5c1-2464-46a2-ab08-6fe9add604c1" containerName="istio-proxy" Jul 22 02:06:46.295617 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.295491 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="184f39b1-faf3-4dcb-8597-ca709e452a5f" containerName="istio-proxy" Jul 22 02:06:46.295617 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.295501 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="582fa5c1-2464-46a2-ab08-6fe9add604c1" containerName="istio-proxy" Jul 22 02:06:46.295617 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.295511 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="184f39b1-faf3-4dcb-8597-ca709e452a5f" containerName="istio-proxy" Jul 22 02:06:46.300080 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.300057 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" Jul 22 02:06:46.304203 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.304181 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 22 02:06:46.305189 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.305167 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 02:06:46.305559 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.305538 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-c62fc15d-serving-cert\"" Jul 22 02:06:46.306035 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.306013 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-49lpj\"" Jul 22 02:06:46.309580 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.309560 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx"] Jul 22 02:06:46.335020 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.334985 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78e7ceae-db66-420f-bca5-bfdfea4aa9a1-openshift-service-ca\") pod \"maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx\" (UID: \"78e7ceae-db66-420f-bca5-bfdfea4aa9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" Jul 22 02:06:46.335144 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.335064 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78e7ceae-db66-420f-bca5-bfdfea4aa9a1-maas-api-tls\") pod \"maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx\" (UID: \"78e7ceae-db66-420f-bca5-bfdfea4aa9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" Jul 22 02:06:46.335144 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.335100 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t72p\" (UniqueName: \"kubernetes.io/projected/78e7ceae-db66-420f-bca5-bfdfea4aa9a1-kube-api-access-6t72p\") pod \"maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx\" (UID: \"78e7ceae-db66-420f-bca5-bfdfea4aa9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" Jul 22 02:06:46.436687 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.436652 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78e7ceae-db66-420f-bca5-bfdfea4aa9a1-openshift-service-ca\") pod \"maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx\" (UID: \"78e7ceae-db66-420f-bca5-bfdfea4aa9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" Jul 22 02:06:46.436867 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.436704 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78e7ceae-db66-420f-bca5-bfdfea4aa9a1-maas-api-tls\") pod \"maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx\" (UID: \"78e7ceae-db66-420f-bca5-bfdfea4aa9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" Jul 22 02:06:46.436867 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.436725 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t72p\" (UniqueName: \"kubernetes.io/projected/78e7ceae-db66-420f-bca5-bfdfea4aa9a1-kube-api-access-6t72p\") pod \"maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx\" (UID: \"78e7ceae-db66-420f-bca5-bfdfea4aa9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" Jul 22 02:06:46.437443 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.437414 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78e7ceae-db66-420f-bca5-bfdfea4aa9a1-openshift-service-ca\") pod \"maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx\" (UID: \"78e7ceae-db66-420f-bca5-bfdfea4aa9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" Jul 22 02:06:46.439093 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.439072 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78e7ceae-db66-420f-bca5-bfdfea4aa9a1-maas-api-tls\") pod \"maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx\" (UID: \"78e7ceae-db66-420f-bca5-bfdfea4aa9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" Jul 22 02:06:46.446154 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.446128 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t72p\" (UniqueName: \"kubernetes.io/projected/78e7ceae-db66-420f-bca5-bfdfea4aa9a1-kube-api-access-6t72p\") pod \"maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx\" (UID: \"78e7ceae-db66-420f-bca5-bfdfea4aa9a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" Jul 22 02:06:46.612583 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.612547 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" Jul 22 02:06:46.749707 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.749679 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx"] Jul 22 02:06:46.752208 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:06:46.752184 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78e7ceae_db66_420f_bca5_bfdfea4aa9a1.slice/crio-9971b14ce4868bf88c1857eeb4131d9b0f73501512e088d3c6456b5164bdc4b2 WatchSource:0}: Error finding container 9971b14ce4868bf88c1857eeb4131d9b0f73501512e088d3c6456b5164bdc4b2: Status 404 returned error can't find the container with id 9971b14ce4868bf88c1857eeb4131d9b0f73501512e088d3c6456b5164bdc4b2 Jul 22 02:06:46.753641 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:46.753619 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:06:47.102074 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:47.102041 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" event={"ID":"78e7ceae-db66-420f-bca5-bfdfea4aa9a1","Type":"ContainerStarted","Data":"9971b14ce4868bf88c1857eeb4131d9b0f73501512e088d3c6456b5164bdc4b2"} Jul 22 02:06:50.121640 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:50.121600 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" event={"ID":"78e7ceae-db66-420f-bca5-bfdfea4aa9a1","Type":"ContainerStarted","Data":"805fb4d523473d6d94a8a59711c9954e486633d08490efe7e2505ae8e1c3369b"} Jul 22 02:06:50.122069 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:50.121780 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" Jul 22 02:06:50.148180 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:50.148117 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" podStartSLOduration=1.726895697 podStartE2EDuration="4.148102015s" totalImagesPullingTime="2.421206318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:06:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:06:50.14716266 +0000 UTC m=+1794.976411007" watchObservedRunningTime="2026-07-22 02:06:50.148102015 +0000 UTC m=+1794.977350387" Jul 22 02:06:55.637713 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:55.637682 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 02:06:55.640305 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:55.640285 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 02:06:55.641958 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:55.641936 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 02:06:55.644741 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:55.644724 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 02:06:56.132106 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:06:56.132079 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" Jul 22 02:07:21.631739 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:21.631700 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx"] Jul 22 02:07:21.632291 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:21.631962 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" podUID="78e7ceae-db66-420f-bca5-bfdfea4aa9a1" containerName="maas-api" containerID="cri-o://805fb4d523473d6d94a8a59711c9954e486633d08490efe7e2505ae8e1c3369b" gracePeriod=30 Jul 22 02:07:21.882530 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:21.882473 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" Jul 22 02:07:21.966148 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:21.966114 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78e7ceae-db66-420f-bca5-bfdfea4aa9a1-openshift-service-ca\") pod \"78e7ceae-db66-420f-bca5-bfdfea4aa9a1\" (UID: \"78e7ceae-db66-420f-bca5-bfdfea4aa9a1\") " Jul 22 02:07:21.966316 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:21.966155 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6t72p\" (UniqueName: \"kubernetes.io/projected/78e7ceae-db66-420f-bca5-bfdfea4aa9a1-kube-api-access-6t72p\") pod \"78e7ceae-db66-420f-bca5-bfdfea4aa9a1\" (UID: \"78e7ceae-db66-420f-bca5-bfdfea4aa9a1\") " Jul 22 02:07:21.966316 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:21.966193 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78e7ceae-db66-420f-bca5-bfdfea4aa9a1-maas-api-tls\") pod \"78e7ceae-db66-420f-bca5-bfdfea4aa9a1\" (UID: \"78e7ceae-db66-420f-bca5-bfdfea4aa9a1\") " Jul 22 02:07:21.966575 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:21.966554 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78e7ceae-db66-420f-bca5-bfdfea4aa9a1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "78e7ceae-db66-420f-bca5-bfdfea4aa9a1" (UID: "78e7ceae-db66-420f-bca5-bfdfea4aa9a1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:07:21.968394 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:21.968375 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78e7ceae-db66-420f-bca5-bfdfea4aa9a1-kube-api-access-6t72p" (OuterVolumeSpecName: "kube-api-access-6t72p") pod "78e7ceae-db66-420f-bca5-bfdfea4aa9a1" (UID: "78e7ceae-db66-420f-bca5-bfdfea4aa9a1"). InnerVolumeSpecName "kube-api-access-6t72p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:07:21.968467 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:21.968447 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78e7ceae-db66-420f-bca5-bfdfea4aa9a1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "78e7ceae-db66-420f-bca5-bfdfea4aa9a1" (UID: "78e7ceae-db66-420f-bca5-bfdfea4aa9a1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:07:22.067264 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:22.067226 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/78e7ceae-db66-420f-bca5-bfdfea4aa9a1-openshift-service-ca\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:07:22.067264 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:22.067257 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6t72p\" (UniqueName: \"kubernetes.io/projected/78e7ceae-db66-420f-bca5-bfdfea4aa9a1-kube-api-access-6t72p\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:07:22.067264 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:22.067267 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/78e7ceae-db66-420f-bca5-bfdfea4aa9a1-maas-api-tls\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:07:22.280849 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:22.280751 2559 generic.go:356] "Generic (PLEG): container finished" podID="78e7ceae-db66-420f-bca5-bfdfea4aa9a1" containerID="805fb4d523473d6d94a8a59711c9954e486633d08490efe7e2505ae8e1c3369b" exitCode=0 Jul 22 02:07:22.280849 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:22.280797 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" Jul 22 02:07:22.280849 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:22.280804 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" event={"ID":"78e7ceae-db66-420f-bca5-bfdfea4aa9a1","Type":"ContainerDied","Data":"805fb4d523473d6d94a8a59711c9954e486633d08490efe7e2505ae8e1c3369b"} Jul 22 02:07:22.281115 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:22.280856 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx" event={"ID":"78e7ceae-db66-420f-bca5-bfdfea4aa9a1","Type":"ContainerDied","Data":"9971b14ce4868bf88c1857eeb4131d9b0f73501512e088d3c6456b5164bdc4b2"} Jul 22 02:07:22.281115 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:22.280883 2559 scope.go:117] "RemoveContainer" containerID="805fb4d523473d6d94a8a59711c9954e486633d08490efe7e2505ae8e1c3369b" Jul 22 02:07:22.291219 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:22.291200 2559 scope.go:117] "RemoveContainer" containerID="805fb4d523473d6d94a8a59711c9954e486633d08490efe7e2505ae8e1c3369b" Jul 22 02:07:22.291505 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:07:22.291486 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"805fb4d523473d6d94a8a59711c9954e486633d08490efe7e2505ae8e1c3369b\": container with ID starting with 805fb4d523473d6d94a8a59711c9954e486633d08490efe7e2505ae8e1c3369b not found: ID does not exist" containerID="805fb4d523473d6d94a8a59711c9954e486633d08490efe7e2505ae8e1c3369b" Jul 22 02:07:22.291587 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:22.291512 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"805fb4d523473d6d94a8a59711c9954e486633d08490efe7e2505ae8e1c3369b"} err="failed to get container status \"805fb4d523473d6d94a8a59711c9954e486633d08490efe7e2505ae8e1c3369b\": rpc error: code = NotFound desc = could not find container \"805fb4d523473d6d94a8a59711c9954e486633d08490efe7e2505ae8e1c3369b\": container with ID starting with 805fb4d523473d6d94a8a59711c9954e486633d08490efe7e2505ae8e1c3369b not found: ID does not exist" Jul 22 02:07:22.305395 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:22.305366 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx"] Jul 22 02:07:22.312688 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:22.312665 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c62fc15d-95cf8bb5d-wcmdx"] Jul 22 02:07:23.626102 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:07:23.626069 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78e7ceae-db66-420f-bca5-bfdfea4aa9a1" path="/var/lib/kubelet/pods/78e7ceae-db66-420f-bca5-bfdfea4aa9a1/volumes" Jul 22 02:09:05.928236 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:05.928202 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m"] Jul 22 02:09:05.928847 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:05.928713 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78e7ceae-db66-420f-bca5-bfdfea4aa9a1" containerName="maas-api" Jul 22 02:09:05.928847 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:05.928733 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="78e7ceae-db66-420f-bca5-bfdfea4aa9a1" containerName="maas-api" Jul 22 02:09:05.928847 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:05.928847 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="78e7ceae-db66-420f-bca5-bfdfea4aa9a1" containerName="maas-api" Jul 22 02:09:05.931806 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:05.931787 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:05.934103 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:05.934087 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-32942ebc-openshift-default-dockercfg-qtnrt\"" Jul 22 02:09:05.958694 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:05.958666 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m"] Jul 22 02:09:06.039671 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.039637 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-istio-data\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.039671 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.039673 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42a4b260-0046-404a-ad9f-41509627ef4d-istio-token\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.039879 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.039698 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-workload-socket\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.039879 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.039721 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-workload-certs\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.039879 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.039786 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-credential-socket\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.039879 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.039838 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-istio-envoy\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.039879 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.039867 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8msd\" (UniqueName: \"kubernetes.io/projected/42a4b260-0046-404a-ad9f-41509627ef4d-kube-api-access-d8msd\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.040046 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.039905 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42a4b260-0046-404a-ad9f-41509627ef4d-istio-podinfo\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.040046 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.039930 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42a4b260-0046-404a-ad9f-41509627ef4d-istiod-ca-cert\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.140591 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.140556 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-istio-data\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.140591 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.140594 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42a4b260-0046-404a-ad9f-41509627ef4d-istio-token\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.140840 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.140620 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-workload-socket\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.140840 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.140650 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-workload-certs\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.140840 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.140687 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-credential-socket\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.140840 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.140722 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-istio-envoy\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.140840 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.140746 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8msd\" (UniqueName: \"kubernetes.io/projected/42a4b260-0046-404a-ad9f-41509627ef4d-kube-api-access-d8msd\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.140840 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.140783 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42a4b260-0046-404a-ad9f-41509627ef4d-istio-podinfo\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.140840 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.140813 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42a4b260-0046-404a-ad9f-41509627ef4d-istiod-ca-cert\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.141251 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.141021 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-istio-data\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.141251 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.141056 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-workload-socket\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.141251 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.141120 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-workload-certs\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.141452 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.141410 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-credential-socket\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.141550 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.141530 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42a4b260-0046-404a-ad9f-41509627ef4d-istiod-ca-cert\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.143429 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.143406 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42a4b260-0046-404a-ad9f-41509627ef4d-istio-podinfo\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.143529 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.143451 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-istio-envoy\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.148962 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.148946 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42a4b260-0046-404a-ad9f-41509627ef4d-istio-token\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.149063 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.149047 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8msd\" (UniqueName: \"kubernetes.io/projected/42a4b260-0046-404a-ad9f-41509627ef4d-kube-api-access-d8msd\") pod \"e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.242221 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.242124 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:06.388530 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.388506 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m"] Jul 22 02:09:06.390779 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:09:06.390745 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42a4b260_0046_404a_ad9f_41509627ef4d.slice/crio-8262458ce5d6e560a680a5bf4667f6d6f55ebe7619e77c2be61c3ecc0c6da149 WatchSource:0}: Error finding container 8262458ce5d6e560a680a5bf4667f6d6f55ebe7619e77c2be61c3ecc0c6da149: Status 404 returned error can't find the container with id 8262458ce5d6e560a680a5bf4667f6d6f55ebe7619e77c2be61c3ecc0c6da149 Jul 22 02:09:06.393020 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.392974 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:09:06.393107 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.393068 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:09:06.393153 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.393115 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:09:06.773850 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.773818 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" event={"ID":"42a4b260-0046-404a-ad9f-41509627ef4d","Type":"ContainerStarted","Data":"dd0daa507a73741511765a5c231fa99ee37afeb25ae6a4f98d4f1bdcda4b9d5d"} Jul 22 02:09:06.774019 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.773864 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" event={"ID":"42a4b260-0046-404a-ad9f-41509627ef4d","Type":"ContainerStarted","Data":"8262458ce5d6e560a680a5bf4667f6d6f55ebe7619e77c2be61c3ecc0c6da149"} Jul 22 02:09:06.796892 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:06.796841 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" podStartSLOduration=1.7968266339999999 podStartE2EDuration="1.796826634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:09:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:09:06.795690192 +0000 UTC m=+1931.624938541" watchObservedRunningTime="2026-07-22 02:09:06.796826634 +0000 UTC m=+1931.626074980" Jul 22 02:09:07.242469 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:07.242429 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:07.247455 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:07.247428 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:07.778889 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:07.778859 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:07.779893 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:07.779875 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:09.003079 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:09.003039 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln"] Jul 22 02:09:09.006915 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:09.006892 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" Jul 22 02:09:09.009420 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:09.009391 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-49lpj\"" Jul 22 02:09:09.009549 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:09.009405 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 02:09:09.009710 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:09.009694 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-32942ebc-serving-cert\"" Jul 22 02:09:09.010184 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:09.010162 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 22 02:09:09.018471 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:09.018446 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln"] Jul 22 02:09:09.069431 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:09.069385 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-openshift-service-ca\") pod \"maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln\" (UID: \"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" Jul 22 02:09:09.069581 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:09.069528 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmnmc\" (UniqueName: \"kubernetes.io/projected/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-kube-api-access-gmnmc\") pod \"maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln\" (UID: \"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" Jul 22 02:09:09.069648 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:09.069591 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-maas-api-tls\") pod \"maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln\" (UID: \"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" Jul 22 02:09:09.170459 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:09.170421 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-openshift-service-ca\") pod \"maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln\" (UID: \"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" Jul 22 02:09:09.170639 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:09.170515 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmnmc\" (UniqueName: \"kubernetes.io/projected/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-kube-api-access-gmnmc\") pod \"maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln\" (UID: \"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" Jul 22 02:09:09.170639 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:09.170577 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-maas-api-tls\") pod \"maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln\" (UID: \"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" Jul 22 02:09:09.170760 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:09:09.170736 2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-serving-cert: secret "maas-api-e2e-mt-32942ebc-serving-cert" not found Jul 22 02:09:09.170827 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:09:09.170816 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-maas-api-tls podName:10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2 nodeName:}" failed. No retries permitted until 2026-07-22 02:09:09.670793975 +0000 UTC m=+1934.500042301 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-maas-api-tls") pod "maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" (UID: "10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2") : secret "maas-api-e2e-mt-32942ebc-serving-cert" not found Jul 22 02:09:09.171083 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:09.171064 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-openshift-service-ca\") pod \"maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln\" (UID: \"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" Jul 22 02:09:09.179357 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:09.179328 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmnmc\" (UniqueName: \"kubernetes.io/projected/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-kube-api-access-gmnmc\") pod \"maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln\" (UID: \"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" Jul 22 02:09:09.675276 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:09.675240 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-maas-api-tls\") pod \"maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln\" (UID: \"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" Jul 22 02:09:09.677654 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:09.677630 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-maas-api-tls\") pod \"maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln\" (UID: \"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" Jul 22 02:09:09.921259 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:09.921223 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" Jul 22 02:09:10.054680 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:10.054653 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln"] Jul 22 02:09:10.057023 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:09:10.056999 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10be2ad6_c62f_4b20_83ce_1d4ff2c8e8f2.slice/crio-a603a7f8f663882702b5ce7b2b5da952cc35a6a31cfef65f90aa4d0bd18c3e74 WatchSource:0}: Error finding container a603a7f8f663882702b5ce7b2b5da952cc35a6a31cfef65f90aa4d0bd18c3e74: Status 404 returned error can't find the container with id a603a7f8f663882702b5ce7b2b5da952cc35a6a31cfef65f90aa4d0bd18c3e74 Jul 22 02:09:10.795463 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:10.795364 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" event={"ID":"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2","Type":"ContainerStarted","Data":"24fd25358637f154d1e3ff7528e83c3db42ae40feff0e6dd3bc03592bf3e47a3"} Jul 22 02:09:10.795463 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:10.795426 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" event={"ID":"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2","Type":"ContainerStarted","Data":"a603a7f8f663882702b5ce7b2b5da952cc35a6a31cfef65f90aa4d0bd18c3e74"} Jul 22 02:09:10.795463 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:10.795456 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" Jul 22 02:09:10.815550 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:10.815507 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" podStartSLOduration=2.401779548 podStartE2EDuration="2.815492486s" totalImagesPullingTime="413.712938ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:09:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:09:10.814465638 +0000 UTC m=+1935.643713987" watchObservedRunningTime="2026-07-22 02:09:10.815492486 +0000 UTC m=+1935.644740833" Jul 22 02:09:16.806780 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:16.806750 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" Jul 22 02:09:22.872478 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:22.871462 2559 patch_prober.go:28] interesting pod/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": read tcp 10.132.0.2:37060->10.132.0.41:15021: read: connection reset by peer" start-of-body= Jul 22 02:09:22.872478 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:22.871538 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" podUID="42a4b260-0046-404a-ad9f-41509627ef4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": read tcp 10.132.0.2:37060->10.132.0.41:15021: read: connection reset by peer" Jul 22 02:09:23.868251 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:23.868220 2559 generic.go:356] "Generic (PLEG): container finished" podID="42a4b260-0046-404a-ad9f-41509627ef4d" containerID="dd0daa507a73741511765a5c231fa99ee37afeb25ae6a4f98d4f1bdcda4b9d5d" exitCode=0 Jul 22 02:09:23.868437 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:23.868297 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" event={"ID":"42a4b260-0046-404a-ad9f-41509627ef4d","Type":"ContainerDied","Data":"dd0daa507a73741511765a5c231fa99ee37afeb25ae6a4f98d4f1bdcda4b9d5d"} Jul 22 02:09:23.868864 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:23.868846 2559 scope.go:117] "RemoveContainer" containerID="dd0daa507a73741511765a5c231fa99ee37afeb25ae6a4f98d4f1bdcda4b9d5d" Jul 22 02:09:23.869925 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:23.869897 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:09:23.870034 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:23.869972 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:09:23.870034 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:23.870006 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:09:24.875555 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:24.875523 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" event={"ID":"42a4b260-0046-404a-ad9f-41509627ef4d","Type":"ContainerStarted","Data":"905869dcbf9c9a012203a1071f441621aaeed017e3d961a73c74ca0f9f661efb"} Jul 22 02:09:25.243207 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:25.243087 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:25.243594 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:25.243569 2559 patch_prober.go:28] interesting pod/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 22 02:09:25.243796 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:25.243769 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" podUID="42a4b260-0046-404a-ad9f-41509627ef4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 22 02:09:26.243043 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:26.243002 2559 patch_prober.go:28] interesting pod/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 22 02:09:26.243673 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:26.243640 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" podUID="42a4b260-0046-404a-ad9f-41509627ef4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 22 02:09:27.242654 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:27.242613 2559 patch_prober.go:28] interesting pod/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 22 02:09:27.242951 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:27.242919 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" podUID="42a4b260-0046-404a-ad9f-41509627ef4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 22 02:09:28.243191 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:28.243133 2559 patch_prober.go:28] interesting pod/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 22 02:09:28.243747 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:28.243210 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" podUID="42a4b260-0046-404a-ad9f-41509627ef4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 22 02:09:29.246283 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:29.246252 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:29.246689 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:29.246625 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:29.247408 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:29.247388 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:31.737769 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:31.737722 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln"] Jul 22 02:09:31.738258 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:31.737977 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" podUID="10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2" containerName="maas-api" containerID="cri-o://24fd25358637f154d1e3ff7528e83c3db42ae40feff0e6dd3bc03592bf3e47a3" gracePeriod=30 Jul 22 02:09:31.801311 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:31.801273 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" podUID="10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.42:8443/health\": dial tcp 10.132.0.42:8443: connect: connection refused" Jul 22 02:09:31.926373 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:31.926305 2559 generic.go:356] "Generic (PLEG): container finished" podID="10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2" containerID="24fd25358637f154d1e3ff7528e83c3db42ae40feff0e6dd3bc03592bf3e47a3" exitCode=0 Jul 22 02:09:31.926546 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:31.926385 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" event={"ID":"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2","Type":"ContainerDied","Data":"24fd25358637f154d1e3ff7528e83c3db42ae40feff0e6dd3bc03592bf3e47a3"} Jul 22 02:09:31.991822 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:31.991757 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" Jul 22 02:09:32.111544 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:32.111507 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-openshift-service-ca\") pod \"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2\" (UID: \"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2\") " Jul 22 02:09:32.111720 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:32.111580 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmnmc\" (UniqueName: \"kubernetes.io/projected/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-kube-api-access-gmnmc\") pod \"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2\" (UID: \"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2\") " Jul 22 02:09:32.111720 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:32.111631 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-maas-api-tls\") pod \"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2\" (UID: \"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2\") " Jul 22 02:09:32.111893 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:32.111869 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2" (UID: "10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:09:32.113746 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:32.113716 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-kube-api-access-gmnmc" (OuterVolumeSpecName: "kube-api-access-gmnmc") pod "10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2" (UID: "10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2"). InnerVolumeSpecName "kube-api-access-gmnmc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:09:32.113860 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:32.113745 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2" (UID: "10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:09:32.212753 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:32.212714 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-maas-api-tls\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:09:32.212753 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:32.212748 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-openshift-service-ca\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:09:32.212753 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:32.212759 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmnmc\" (UniqueName: \"kubernetes.io/projected/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2-kube-api-access-gmnmc\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:09:32.932215 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:32.932184 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" Jul 22 02:09:32.932730 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:32.932221 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln" event={"ID":"10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2","Type":"ContainerDied","Data":"a603a7f8f663882702b5ce7b2b5da952cc35a6a31cfef65f90aa4d0bd18c3e74"} Jul 22 02:09:32.932730 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:32.932280 2559 scope.go:117] "RemoveContainer" containerID="24fd25358637f154d1e3ff7528e83c3db42ae40feff0e6dd3bc03592bf3e47a3" Jul 22 02:09:32.954354 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:32.954314 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln"] Jul 22 02:09:32.959035 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:32.959010 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-32942ebc-5cf8b7d654-6kvln"] Jul 22 02:09:33.625631 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:33.625594 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2" path="/var/lib/kubelet/pods/10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2/volumes" Jul 22 02:09:42.853767 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:42.853733 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m"] Jul 22 02:09:42.854267 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:42.854001 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" podUID="42a4b260-0046-404a-ad9f-41509627ef4d" containerName="istio-proxy" containerID="cri-o://905869dcbf9c9a012203a1071f441621aaeed017e3d961a73c74ca0f9f661efb" gracePeriod=30 Jul 22 02:09:44.249169 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:44.249133 2559 patch_prober.go:28] interesting pod/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 02:09:44.249568 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:44.249194 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" podUID="42a4b260-0046-404a-ad9f-41509627ef4d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 02:09:48.009488 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.009450 2559 generic.go:356] "Generic (PLEG): container finished" podID="42a4b260-0046-404a-ad9f-41509627ef4d" containerID="905869dcbf9c9a012203a1071f441621aaeed017e3d961a73c74ca0f9f661efb" exitCode=0 Jul 22 02:09:48.009488 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.009488 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" event={"ID":"42a4b260-0046-404a-ad9f-41509627ef4d","Type":"ContainerDied","Data":"905869dcbf9c9a012203a1071f441621aaeed017e3d961a73c74ca0f9f661efb"} Jul 22 02:09:48.010057 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.009535 2559 scope.go:117] "RemoveContainer" containerID="dd0daa507a73741511765a5c231fa99ee37afeb25ae6a4f98d4f1bdcda4b9d5d" Jul 22 02:09:48.115152 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.115095 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:48.147528 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.147500 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-workload-certs\") pod \"42a4b260-0046-404a-ad9f-41509627ef4d\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " Jul 22 02:09:48.147681 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.147554 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-workload-socket\") pod \"42a4b260-0046-404a-ad9f-41509627ef4d\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " Jul 22 02:09:48.147681 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.147584 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42a4b260-0046-404a-ad9f-41509627ef4d-istio-podinfo\") pod \"42a4b260-0046-404a-ad9f-41509627ef4d\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " Jul 22 02:09:48.147681 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.147631 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8msd\" (UniqueName: \"kubernetes.io/projected/42a4b260-0046-404a-ad9f-41509627ef4d-kube-api-access-d8msd\") pod \"42a4b260-0046-404a-ad9f-41509627ef4d\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " Jul 22 02:09:48.147681 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.147659 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42a4b260-0046-404a-ad9f-41509627ef4d-istiod-ca-cert\") pod \"42a4b260-0046-404a-ad9f-41509627ef4d\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " Jul 22 02:09:48.147904 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.147706 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-istio-data\") pod \"42a4b260-0046-404a-ad9f-41509627ef4d\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " Jul 22 02:09:48.147904 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.147759 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-credential-socket\") pod \"42a4b260-0046-404a-ad9f-41509627ef4d\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " Jul 22 02:09:48.147904 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.147790 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "42a4b260-0046-404a-ad9f-41509627ef4d" (UID: "42a4b260-0046-404a-ad9f-41509627ef4d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:09:48.147904 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.147825 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-istio-envoy\") pod \"42a4b260-0046-404a-ad9f-41509627ef4d\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " Jul 22 02:09:48.147904 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.147862 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42a4b260-0046-404a-ad9f-41509627ef4d-istio-token\") pod \"42a4b260-0046-404a-ad9f-41509627ef4d\" (UID: \"42a4b260-0046-404a-ad9f-41509627ef4d\") " Jul 22 02:09:48.147904 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.147888 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "42a4b260-0046-404a-ad9f-41509627ef4d" (UID: "42a4b260-0046-404a-ad9f-41509627ef4d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:09:48.148223 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.148021 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42a4b260-0046-404a-ad9f-41509627ef4d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "42a4b260-0046-404a-ad9f-41509627ef4d" (UID: "42a4b260-0046-404a-ad9f-41509627ef4d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:09:48.148223 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.148087 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "42a4b260-0046-404a-ad9f-41509627ef4d" (UID: "42a4b260-0046-404a-ad9f-41509627ef4d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:09:48.148330 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.148221 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-workload-certs\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:09:48.148330 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.148240 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-workload-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:09:48.148330 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.148254 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42a4b260-0046-404a-ad9f-41509627ef4d-istiod-ca-cert\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:09:48.148330 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.148269 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-credential-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:09:48.148670 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.148642 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-istio-data" (OuterVolumeSpecName: "istio-data") pod "42a4b260-0046-404a-ad9f-41509627ef4d" (UID: "42a4b260-0046-404a-ad9f-41509627ef4d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:09:48.149898 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.149874 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42a4b260-0046-404a-ad9f-41509627ef4d-kube-api-access-d8msd" (OuterVolumeSpecName: "kube-api-access-d8msd") pod "42a4b260-0046-404a-ad9f-41509627ef4d" (UID: "42a4b260-0046-404a-ad9f-41509627ef4d"). InnerVolumeSpecName "kube-api-access-d8msd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:09:48.150193 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.150157 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "42a4b260-0046-404a-ad9f-41509627ef4d" (UID: "42a4b260-0046-404a-ad9f-41509627ef4d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:09:48.150279 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.150249 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42a4b260-0046-404a-ad9f-41509627ef4d-istio-token" (OuterVolumeSpecName: "istio-token") pod "42a4b260-0046-404a-ad9f-41509627ef4d" (UID: "42a4b260-0046-404a-ad9f-41509627ef4d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:09:48.150279 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.150261 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/42a4b260-0046-404a-ad9f-41509627ef4d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "42a4b260-0046-404a-ad9f-41509627ef4d" (UID: "42a4b260-0046-404a-ad9f-41509627ef4d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:09:48.249675 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.249644 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42a4b260-0046-404a-ad9f-41509627ef4d-istio-podinfo\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:09:48.249675 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.249668 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8msd\" (UniqueName: \"kubernetes.io/projected/42a4b260-0046-404a-ad9f-41509627ef4d-kube-api-access-d8msd\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:09:48.249675 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.249677 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-istio-data\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:09:48.249882 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.249688 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42a4b260-0046-404a-ad9f-41509627ef4d-istio-envoy\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:09:48.249882 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:48.249697 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42a4b260-0046-404a-ad9f-41509627ef4d-istio-token\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:09:49.015702 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:49.015662 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" event={"ID":"42a4b260-0046-404a-ad9f-41509627ef4d","Type":"ContainerDied","Data":"8262458ce5d6e560a680a5bf4667f6d6f55ebe7619e77c2be61c3ecc0c6da149"} Jul 22 02:09:49.015702 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:49.015690 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m" Jul 22 02:09:49.016180 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:49.015762 2559 scope.go:117] "RemoveContainer" containerID="905869dcbf9c9a012203a1071f441621aaeed017e3d961a73c74ca0f9f661efb" Jul 22 02:09:49.039804 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:49.039776 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m"] Jul 22 02:09:49.047052 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:49.047026 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-32942ebc-openshift-default-5956869ff9-lkh6m"] Jul 22 02:09:49.625498 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:09:49.625465 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42a4b260-0046-404a-ad9f-41509627ef4d" path="/var/lib/kubelet/pods/42a4b260-0046-404a-ad9f-41509627ef4d/volumes" Jul 22 02:10:03.066640 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.066605 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff"] Jul 22 02:10:03.067026 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.066961 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42a4b260-0046-404a-ad9f-41509627ef4d" containerName="istio-proxy" Jul 22 02:10:03.067026 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.066972 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="42a4b260-0046-404a-ad9f-41509627ef4d" containerName="istio-proxy" Jul 22 02:10:03.067026 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.066995 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2" containerName="maas-api" Jul 22 02:10:03.067026 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.067001 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2" containerName="maas-api" Jul 22 02:10:03.067158 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.067064 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="10be2ad6-c62f-4b20-83ce-1d4ff2c8e8f2" containerName="maas-api" Jul 22 02:10:03.067158 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.067077 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="42a4b260-0046-404a-ad9f-41509627ef4d" containerName="istio-proxy" Jul 22 02:10:03.067158 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.067084 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="42a4b260-0046-404a-ad9f-41509627ef4d" containerName="istio-proxy" Jul 22 02:10:03.070019 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.070001 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.072565 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.072543 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-fb9ad51b-openshift-default-dockercfg-5pdjk\"" Jul 22 02:10:03.086817 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.086789 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff"] Jul 22 02:10:03.188291 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.188249 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-data\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.188291 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.188291 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-token\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.188559 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.188446 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-workload-socket\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.188559 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.188485 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-envoy\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.188559 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.188506 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-credential-socket\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.188559 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.188520 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-workload-certs\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.188559 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.188553 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-podinfo\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.188747 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.188616 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istiod-ca-cert\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.188747 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.188636 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48x57\" (UniqueName: \"kubernetes.io/projected/311b2b28-94f0-4ce9-b7da-0f306ec258c4-kube-api-access-48x57\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.289112 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.289069 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istiod-ca-cert\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.289112 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.289117 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48x57\" (UniqueName: \"kubernetes.io/projected/311b2b28-94f0-4ce9-b7da-0f306ec258c4-kube-api-access-48x57\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.289412 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.289162 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-data\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.289412 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.289186 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-token\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.289412 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.289284 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-workload-socket\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.289412 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.289306 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-envoy\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.289412 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.289337 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-credential-socket\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.289412 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.289380 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-workload-certs\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.289723 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.289425 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-podinfo\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.289723 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.289588 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-data\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.289834 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.289817 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-credential-socket\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.289893 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.289869 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-workload-socket\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.289936 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.289898 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istiod-ca-cert\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.290114 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.290095 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-workload-certs\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.291925 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.291898 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-podinfo\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.292051 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.291985 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-envoy\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.298774 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.298750 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-token\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.298992 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.298968 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48x57\" (UniqueName: \"kubernetes.io/projected/311b2b28-94f0-4ce9-b7da-0f306ec258c4-kube-api-access-48x57\") pod \"e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.379939 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.379893 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:03.529748 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.529724 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff"] Jul 22 02:10:03.532313 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:10:03.532279 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod311b2b28_94f0_4ce9_b7da_0f306ec258c4.slice/crio-fddc51d1b07b90e9d03c06bfa2beadc916e7e08b26463c2f45328afddaa2a8b5 WatchSource:0}: Error finding container fddc51d1b07b90e9d03c06bfa2beadc916e7e08b26463c2f45328afddaa2a8b5: Status 404 returned error can't find the container with id fddc51d1b07b90e9d03c06bfa2beadc916e7e08b26463c2f45328afddaa2a8b5 Jul 22 02:10:03.534533 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.534489 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:10:03.534629 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.534582 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:10:03.534629 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:03.534623 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:10:04.087902 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:04.087869 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" event={"ID":"311b2b28-94f0-4ce9-b7da-0f306ec258c4","Type":"ContainerStarted","Data":"776f07d52f99d073631864051961627c31ecfd4b47968d9d87cbabd816eff738"} Jul 22 02:10:04.088542 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:04.087927 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" event={"ID":"311b2b28-94f0-4ce9-b7da-0f306ec258c4","Type":"ContainerStarted","Data":"fddc51d1b07b90e9d03c06bfa2beadc916e7e08b26463c2f45328afddaa2a8b5"} Jul 22 02:10:04.109580 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:04.109519 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" podStartSLOduration=1.109496022 podStartE2EDuration="1.109496022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:10:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:10:04.108161709 +0000 UTC m=+1988.937410056" watchObservedRunningTime="2026-07-22 02:10:04.109496022 +0000 UTC m=+1988.938744377" Jul 22 02:10:04.380982 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:04.380884 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:04.385948 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:04.385923 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:05.093377 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:05.093328 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:05.094463 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:05.094447 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:06.149705 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.149671 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg"] Jul 22 02:10:06.150092 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.150075 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42a4b260-0046-404a-ad9f-41509627ef4d" containerName="istio-proxy" Jul 22 02:10:06.150092 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.150085 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="42a4b260-0046-404a-ad9f-41509627ef4d" containerName="istio-proxy" Jul 22 02:10:06.153462 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.153438 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" Jul 22 02:10:06.155939 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.155916 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 22 02:10:06.156076 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.155916 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 02:10:06.156674 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.156653 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-fb9ad51b-serving-cert\"" Jul 22 02:10:06.156674 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.156663 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-49lpj\"" Jul 22 02:10:06.163653 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.163626 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg"] Jul 22 02:10:06.219788 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.219747 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9r6v\" (UniqueName: \"kubernetes.io/projected/bf7b2888-5806-44bb-8dbc-c5420c737606-kube-api-access-r9r6v\") pod \"maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg\" (UID: \"bf7b2888-5806-44bb-8dbc-c5420c737606\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" Jul 22 02:10:06.220010 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.219870 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf7b2888-5806-44bb-8dbc-c5420c737606-maas-api-tls\") pod \"maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg\" (UID: \"bf7b2888-5806-44bb-8dbc-c5420c737606\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" Jul 22 02:10:06.220098 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.220016 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf7b2888-5806-44bb-8dbc-c5420c737606-openshift-service-ca\") pod \"maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg\" (UID: \"bf7b2888-5806-44bb-8dbc-c5420c737606\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" Jul 22 02:10:06.320930 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.320889 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf7b2888-5806-44bb-8dbc-c5420c737606-maas-api-tls\") pod \"maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg\" (UID: \"bf7b2888-5806-44bb-8dbc-c5420c737606\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" Jul 22 02:10:06.321129 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.321046 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf7b2888-5806-44bb-8dbc-c5420c737606-openshift-service-ca\") pod \"maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg\" (UID: \"bf7b2888-5806-44bb-8dbc-c5420c737606\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" Jul 22 02:10:06.321129 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.321109 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9r6v\" (UniqueName: \"kubernetes.io/projected/bf7b2888-5806-44bb-8dbc-c5420c737606-kube-api-access-r9r6v\") pod \"maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg\" (UID: \"bf7b2888-5806-44bb-8dbc-c5420c737606\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" Jul 22 02:10:06.321779 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.321758 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf7b2888-5806-44bb-8dbc-c5420c737606-openshift-service-ca\") pod \"maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg\" (UID: \"bf7b2888-5806-44bb-8dbc-c5420c737606\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" Jul 22 02:10:06.323486 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.323458 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf7b2888-5806-44bb-8dbc-c5420c737606-maas-api-tls\") pod \"maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg\" (UID: \"bf7b2888-5806-44bb-8dbc-c5420c737606\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" Jul 22 02:10:06.330617 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.330571 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9r6v\" (UniqueName: \"kubernetes.io/projected/bf7b2888-5806-44bb-8dbc-c5420c737606-kube-api-access-r9r6v\") pod \"maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg\" (UID: \"bf7b2888-5806-44bb-8dbc-c5420c737606\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" Jul 22 02:10:06.467316 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.467209 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" Jul 22 02:10:06.609313 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:06.609286 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg"] Jul 22 02:10:06.611375 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:10:06.611316 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf7b2888_5806_44bb_8dbc_c5420c737606.slice/crio-d96449d0c8e0fdc6ef9c9c008249815fdef12dd6bf9a050d1a50d3c17c0e0f89 WatchSource:0}: Error finding container d96449d0c8e0fdc6ef9c9c008249815fdef12dd6bf9a050d1a50d3c17c0e0f89: Status 404 returned error can't find the container with id d96449d0c8e0fdc6ef9c9c008249815fdef12dd6bf9a050d1a50d3c17c0e0f89 Jul 22 02:10:07.105181 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:07.105147 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" event={"ID":"bf7b2888-5806-44bb-8dbc-c5420c737606","Type":"ContainerStarted","Data":"d96449d0c8e0fdc6ef9c9c008249815fdef12dd6bf9a050d1a50d3c17c0e0f89"} Jul 22 02:10:08.112024 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:08.111989 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" event={"ID":"bf7b2888-5806-44bb-8dbc-c5420c737606","Type":"ContainerStarted","Data":"63c11455e4fa33f961197d08d279314ad32c881c57dd4429612b92bf6a5f7c7e"} Jul 22 02:10:08.112528 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:08.112115 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" Jul 22 02:10:08.131992 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:08.131932 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" podStartSLOduration=1.6796370440000001 podStartE2EDuration="2.131915138s" totalImagesPullingTime="452.278094ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:10:08.129285475 +0000 UTC m=+1992.958533823" watchObservedRunningTime="2026-07-22 02:10:08.131915138 +0000 UTC m=+1992.961163485" Jul 22 02:10:14.122408 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:14.122376 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" Jul 22 02:10:20.614095 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:20.612704 2559 patch_prober.go:28] interesting pod/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": read tcp 10.132.0.2:44332->10.132.0.43:15021: read: connection reset by peer" start-of-body= Jul 22 02:10:20.614095 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:20.612773 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" podUID="311b2b28-94f0-4ce9-b7da-0f306ec258c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": read tcp 10.132.0.2:44332->10.132.0.43:15021: read: connection reset by peer" Jul 22 02:10:21.189135 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:21.189100 2559 generic.go:356] "Generic (PLEG): container finished" podID="311b2b28-94f0-4ce9-b7da-0f306ec258c4" containerID="776f07d52f99d073631864051961627c31ecfd4b47968d9d87cbabd816eff738" exitCode=0 Jul 22 02:10:21.189304 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:21.189175 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" event={"ID":"311b2b28-94f0-4ce9-b7da-0f306ec258c4","Type":"ContainerDied","Data":"776f07d52f99d073631864051961627c31ecfd4b47968d9d87cbabd816eff738"} Jul 22 02:10:21.189622 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:21.189609 2559 scope.go:117] "RemoveContainer" containerID="776f07d52f99d073631864051961627c31ecfd4b47968d9d87cbabd816eff738" Jul 22 02:10:21.190668 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:21.190638 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:10:21.190771 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:21.190707 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:10:21.190771 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:21.190737 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:10:22.196200 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:22.196165 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" event={"ID":"311b2b28-94f0-4ce9-b7da-0f306ec258c4","Type":"ContainerStarted","Data":"7c676b1bc1344eccde1f747e9aff3769bf02824677fbd9822592361a1d5ee5d1"} Jul 22 02:10:22.380523 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:22.380488 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:22.380749 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:22.380653 2559 patch_prober.go:28] interesting pod/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 22 02:10:22.380749 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:22.380707 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" podUID="311b2b28-94f0-4ce9-b7da-0f306ec258c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 22 02:10:23.380842 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:23.380795 2559 patch_prober.go:28] interesting pod/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 22 02:10:23.381384 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:23.380884 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" podUID="311b2b28-94f0-4ce9-b7da-0f306ec258c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 22 02:10:24.380679 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:24.380642 2559 patch_prober.go:28] interesting pod/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 22 02:10:24.380854 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:24.380703 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" podUID="311b2b28-94f0-4ce9-b7da-0f306ec258c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 22 02:10:25.380683 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:25.380641 2559 patch_prober.go:28] interesting pod/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 22 02:10:25.381049 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:25.380700 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" podUID="311b2b28-94f0-4ce9-b7da-0f306ec258c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 22 02:10:26.380790 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:26.380754 2559 patch_prober.go:28] interesting pod/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 22 02:10:26.381193 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:26.380806 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" podUID="311b2b28-94f0-4ce9-b7da-0f306ec258c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 22 02:10:27.384248 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:27.384215 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:27.384664 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:27.384592 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:27.385313 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:27.385295 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:10:49.446949 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:49.446917 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg"] Jul 22 02:10:49.447446 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:49.447116 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" podUID="bf7b2888-5806-44bb-8dbc-c5420c737606" containerName="maas-api" containerID="cri-o://63c11455e4fa33f961197d08d279314ad32c881c57dd4429612b92bf6a5f7c7e" gracePeriod=30 Jul 22 02:10:49.700054 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:49.699991 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" Jul 22 02:10:49.844030 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:49.843991 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf7b2888-5806-44bb-8dbc-c5420c737606-maas-api-tls\") pod \"bf7b2888-5806-44bb-8dbc-c5420c737606\" (UID: \"bf7b2888-5806-44bb-8dbc-c5420c737606\") " Jul 22 02:10:49.844211 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:49.844132 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf7b2888-5806-44bb-8dbc-c5420c737606-openshift-service-ca\") pod \"bf7b2888-5806-44bb-8dbc-c5420c737606\" (UID: \"bf7b2888-5806-44bb-8dbc-c5420c737606\") " Jul 22 02:10:49.844211 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:49.844155 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9r6v\" (UniqueName: \"kubernetes.io/projected/bf7b2888-5806-44bb-8dbc-c5420c737606-kube-api-access-r9r6v\") pod \"bf7b2888-5806-44bb-8dbc-c5420c737606\" (UID: \"bf7b2888-5806-44bb-8dbc-c5420c737606\") " Jul 22 02:10:49.844455 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:49.844430 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf7b2888-5806-44bb-8dbc-c5420c737606-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bf7b2888-5806-44bb-8dbc-c5420c737606" (UID: "bf7b2888-5806-44bb-8dbc-c5420c737606"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:10:49.846140 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:49.846119 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf7b2888-5806-44bb-8dbc-c5420c737606-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bf7b2888-5806-44bb-8dbc-c5420c737606" (UID: "bf7b2888-5806-44bb-8dbc-c5420c737606"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:10:49.846257 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:49.846238 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf7b2888-5806-44bb-8dbc-c5420c737606-kube-api-access-r9r6v" (OuterVolumeSpecName: "kube-api-access-r9r6v") pod "bf7b2888-5806-44bb-8dbc-c5420c737606" (UID: "bf7b2888-5806-44bb-8dbc-c5420c737606"). InnerVolumeSpecName "kube-api-access-r9r6v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:10:49.945707 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:49.945679 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf7b2888-5806-44bb-8dbc-c5420c737606-openshift-service-ca\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:10:49.945707 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:49.945702 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9r6v\" (UniqueName: \"kubernetes.io/projected/bf7b2888-5806-44bb-8dbc-c5420c737606-kube-api-access-r9r6v\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:10:49.945707 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:49.945714 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf7b2888-5806-44bb-8dbc-c5420c737606-maas-api-tls\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:10:50.343015 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:50.342977 2559 generic.go:356] "Generic (PLEG): container finished" podID="bf7b2888-5806-44bb-8dbc-c5420c737606" containerID="63c11455e4fa33f961197d08d279314ad32c881c57dd4429612b92bf6a5f7c7e" exitCode=0 Jul 22 02:10:50.343212 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:50.343033 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" Jul 22 02:10:50.343212 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:50.343066 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" event={"ID":"bf7b2888-5806-44bb-8dbc-c5420c737606","Type":"ContainerDied","Data":"63c11455e4fa33f961197d08d279314ad32c881c57dd4429612b92bf6a5f7c7e"} Jul 22 02:10:50.343212 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:50.343136 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg" event={"ID":"bf7b2888-5806-44bb-8dbc-c5420c737606","Type":"ContainerDied","Data":"d96449d0c8e0fdc6ef9c9c008249815fdef12dd6bf9a050d1a50d3c17c0e0f89"} Jul 22 02:10:50.343212 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:50.343158 2559 scope.go:117] "RemoveContainer" containerID="63c11455e4fa33f961197d08d279314ad32c881c57dd4429612b92bf6a5f7c7e" Jul 22 02:10:50.352242 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:50.352228 2559 scope.go:117] "RemoveContainer" containerID="63c11455e4fa33f961197d08d279314ad32c881c57dd4429612b92bf6a5f7c7e" Jul 22 02:10:50.352538 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:10:50.352520 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63c11455e4fa33f961197d08d279314ad32c881c57dd4429612b92bf6a5f7c7e\": container with ID starting with 63c11455e4fa33f961197d08d279314ad32c881c57dd4429612b92bf6a5f7c7e not found: ID does not exist" containerID="63c11455e4fa33f961197d08d279314ad32c881c57dd4429612b92bf6a5f7c7e" Jul 22 02:10:50.352723 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:50.352545 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63c11455e4fa33f961197d08d279314ad32c881c57dd4429612b92bf6a5f7c7e"} err="failed to get container status \"63c11455e4fa33f961197d08d279314ad32c881c57dd4429612b92bf6a5f7c7e\": rpc error: code = NotFound desc = could not find container \"63c11455e4fa33f961197d08d279314ad32c881c57dd4429612b92bf6a5f7c7e\": container with ID starting with 63c11455e4fa33f961197d08d279314ad32c881c57dd4429612b92bf6a5f7c7e not found: ID does not exist" Jul 22 02:10:50.363886 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:50.363864 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg"] Jul 22 02:10:50.367667 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:50.367645 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fb9ad51b-849b7c58db-2mtxg"] Jul 22 02:10:51.625791 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:51.625760 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf7b2888-5806-44bb-8dbc-c5420c737606" path="/var/lib/kubelet/pods/bf7b2888-5806-44bb-8dbc-c5420c737606/volumes" Jul 22 02:10:55.749160 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:10:55.749129 2559 scope.go:117] "RemoveContainer" containerID="e40010ca7f3389080dc954dd2a7453583651a42ccd5894987bc2b302a9869f81" Jul 22 02:11:44.980648 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:44.980612 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff"] Jul 22 02:11:44.981224 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:44.980839 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" podUID="311b2b28-94f0-4ce9-b7da-0f306ec258c4" containerName="istio-proxy" containerID="cri-o://7c676b1bc1344eccde1f747e9aff3769bf02824677fbd9822592361a1d5ee5d1" gracePeriod=30 Jul 22 02:11:50.229922 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.229894 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:11:50.376062 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.376025 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-workload-socket\") pod \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " Jul 22 02:11:50.376247 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.376085 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-envoy\") pod \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " Jul 22 02:11:50.376247 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.376122 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-podinfo\") pod \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " Jul 22 02:11:50.376247 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.376147 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-data\") pod \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " Jul 22 02:11:50.376247 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.376179 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-credential-socket\") pod \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " Jul 22 02:11:50.376515 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.376302 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "311b2b28-94f0-4ce9-b7da-0f306ec258c4" (UID: "311b2b28-94f0-4ce9-b7da-0f306ec258c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:11:50.376515 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.376326 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-workload-certs\") pod \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " Jul 22 02:11:50.376515 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.376393 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istiod-ca-cert\") pod \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " Jul 22 02:11:50.376515 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.376425 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-token\") pod \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " Jul 22 02:11:50.376515 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.376466 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48x57\" (UniqueName: \"kubernetes.io/projected/311b2b28-94f0-4ce9-b7da-0f306ec258c4-kube-api-access-48x57\") pod \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\" (UID: \"311b2b28-94f0-4ce9-b7da-0f306ec258c4\") " Jul 22 02:11:50.376515 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.376487 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "311b2b28-94f0-4ce9-b7da-0f306ec258c4" (UID: "311b2b28-94f0-4ce9-b7da-0f306ec258c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:11:50.376828 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.376555 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "311b2b28-94f0-4ce9-b7da-0f306ec258c4" (UID: "311b2b28-94f0-4ce9-b7da-0f306ec258c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:11:50.376828 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.376765 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-credential-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:11:50.376828 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.376778 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-workload-certs\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:11:50.376828 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.376787 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-workload-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:11:50.377032 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.376881 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "311b2b28-94f0-4ce9-b7da-0f306ec258c4" (UID: "311b2b28-94f0-4ce9-b7da-0f306ec258c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:11:50.377225 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.377193 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-data" (OuterVolumeSpecName: "istio-data") pod "311b2b28-94f0-4ce9-b7da-0f306ec258c4" (UID: "311b2b28-94f0-4ce9-b7da-0f306ec258c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:11:50.378487 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.378459 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-token" (OuterVolumeSpecName: "istio-token") pod "311b2b28-94f0-4ce9-b7da-0f306ec258c4" (UID: "311b2b28-94f0-4ce9-b7da-0f306ec258c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:11:50.378591 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.378505 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "311b2b28-94f0-4ce9-b7da-0f306ec258c4" (UID: "311b2b28-94f0-4ce9-b7da-0f306ec258c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:11:50.378591 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.378524 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "311b2b28-94f0-4ce9-b7da-0f306ec258c4" (UID: "311b2b28-94f0-4ce9-b7da-0f306ec258c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:11:50.378591 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.378535 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/311b2b28-94f0-4ce9-b7da-0f306ec258c4-kube-api-access-48x57" (OuterVolumeSpecName: "kube-api-access-48x57") pod "311b2b28-94f0-4ce9-b7da-0f306ec258c4" (UID: "311b2b28-94f0-4ce9-b7da-0f306ec258c4"). InnerVolumeSpecName "kube-api-access-48x57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:11:50.478419 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.478386 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istiod-ca-cert\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:11:50.478419 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.478421 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-token\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:11:50.478666 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.478436 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48x57\" (UniqueName: \"kubernetes.io/projected/311b2b28-94f0-4ce9-b7da-0f306ec258c4-kube-api-access-48x57\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:11:50.478666 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.478452 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-envoy\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:11:50.478666 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.478465 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-podinfo\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:11:50.478666 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.478477 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/311b2b28-94f0-4ce9-b7da-0f306ec258c4-istio-data\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:11:50.636736 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.636650 2559 generic.go:356] "Generic (PLEG): container finished" podID="311b2b28-94f0-4ce9-b7da-0f306ec258c4" containerID="7c676b1bc1344eccde1f747e9aff3769bf02824677fbd9822592361a1d5ee5d1" exitCode=0 Jul 22 02:11:50.636736 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.636690 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" event={"ID":"311b2b28-94f0-4ce9-b7da-0f306ec258c4","Type":"ContainerDied","Data":"7c676b1bc1344eccde1f747e9aff3769bf02824677fbd9822592361a1d5ee5d1"} Jul 22 02:11:50.636736 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.636694 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" Jul 22 02:11:50.636736 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.636734 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff" event={"ID":"311b2b28-94f0-4ce9-b7da-0f306ec258c4","Type":"ContainerDied","Data":"fddc51d1b07b90e9d03c06bfa2beadc916e7e08b26463c2f45328afddaa2a8b5"} Jul 22 02:11:50.637002 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.636764 2559 scope.go:117] "RemoveContainer" containerID="7c676b1bc1344eccde1f747e9aff3769bf02824677fbd9822592361a1d5ee5d1" Jul 22 02:11:50.646389 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.646368 2559 scope.go:117] "RemoveContainer" containerID="776f07d52f99d073631864051961627c31ecfd4b47968d9d87cbabd816eff738" Jul 22 02:11:50.656553 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.656534 2559 scope.go:117] "RemoveContainer" containerID="7c676b1bc1344eccde1f747e9aff3769bf02824677fbd9822592361a1d5ee5d1" Jul 22 02:11:50.656829 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:11:50.656809 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c676b1bc1344eccde1f747e9aff3769bf02824677fbd9822592361a1d5ee5d1\": container with ID starting with 7c676b1bc1344eccde1f747e9aff3769bf02824677fbd9822592361a1d5ee5d1 not found: ID does not exist" containerID="7c676b1bc1344eccde1f747e9aff3769bf02824677fbd9822592361a1d5ee5d1" Jul 22 02:11:50.656877 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.656840 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c676b1bc1344eccde1f747e9aff3769bf02824677fbd9822592361a1d5ee5d1"} err="failed to get container status \"7c676b1bc1344eccde1f747e9aff3769bf02824677fbd9822592361a1d5ee5d1\": rpc error: code = NotFound desc = could not find container \"7c676b1bc1344eccde1f747e9aff3769bf02824677fbd9822592361a1d5ee5d1\": container with ID starting with 7c676b1bc1344eccde1f747e9aff3769bf02824677fbd9822592361a1d5ee5d1 not found: ID does not exist" Jul 22 02:11:50.656877 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.656857 2559 scope.go:117] "RemoveContainer" containerID="776f07d52f99d073631864051961627c31ecfd4b47968d9d87cbabd816eff738" Jul 22 02:11:50.657112 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:11:50.657093 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"776f07d52f99d073631864051961627c31ecfd4b47968d9d87cbabd816eff738\": container with ID starting with 776f07d52f99d073631864051961627c31ecfd4b47968d9d87cbabd816eff738 not found: ID does not exist" containerID="776f07d52f99d073631864051961627c31ecfd4b47968d9d87cbabd816eff738" Jul 22 02:11:50.657166 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.657117 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"776f07d52f99d073631864051961627c31ecfd4b47968d9d87cbabd816eff738"} err="failed to get container status \"776f07d52f99d073631864051961627c31ecfd4b47968d9d87cbabd816eff738\": rpc error: code = NotFound desc = could not find container \"776f07d52f99d073631864051961627c31ecfd4b47968d9d87cbabd816eff738\": container with ID starting with 776f07d52f99d073631864051961627c31ecfd4b47968d9d87cbabd816eff738 not found: ID does not exist" Jul 22 02:11:50.661589 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.661568 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff"] Jul 22 02:11:50.668091 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:50.668070 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-fb9ad51b-openshift-default-589984c487-2ggff"] Jul 22 02:11:51.625291 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:51.625256 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="311b2b28-94f0-4ce9-b7da-0f306ec258c4" path="/var/lib/kubelet/pods/311b2b28-94f0-4ce9-b7da-0f306ec258c4/volumes" Jul 22 02:11:55.667381 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:55.667355 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 02:11:55.671121 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:55.671090 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 02:11:55.671624 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:55.671608 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 02:11:55.675660 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:55.675627 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 02:11:55.770979 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:11:55.770955 2559 scope.go:117] "RemoveContainer" containerID="203b79e456986172134a4c96ebd245fc6262c454fba35a035d1eeb135a0dea0e" Jul 22 02:12:42.094311 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.094276 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s"] Jul 22 02:12:42.094833 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.094684 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="311b2b28-94f0-4ce9-b7da-0f306ec258c4" containerName="istio-proxy" Jul 22 02:12:42.094833 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.094698 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="311b2b28-94f0-4ce9-b7da-0f306ec258c4" containerName="istio-proxy" Jul 22 02:12:42.094833 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.094711 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf7b2888-5806-44bb-8dbc-c5420c737606" containerName="maas-api" Jul 22 02:12:42.094833 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.094716 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf7b2888-5806-44bb-8dbc-c5420c737606" containerName="maas-api" Jul 22 02:12:42.094833 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.094735 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="311b2b28-94f0-4ce9-b7da-0f306ec258c4" containerName="istio-proxy" Jul 22 02:12:42.094833 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.094742 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="311b2b28-94f0-4ce9-b7da-0f306ec258c4" containerName="istio-proxy" Jul 22 02:12:42.094833 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.094821 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf7b2888-5806-44bb-8dbc-c5420c737606" containerName="maas-api" Jul 22 02:12:42.094833 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.094833 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="311b2b28-94f0-4ce9-b7da-0f306ec258c4" containerName="istio-proxy" Jul 22 02:12:42.098022 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.098004 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.100391 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.100367 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-b0ac34-openshift-default-dockercfg-xq6jf\"" Jul 22 02:12:42.110285 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.110261 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s"] Jul 22 02:12:42.255300 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.255246 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-envoy\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.255506 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.255322 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-credential-socket\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.255506 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.255377 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj525\" (UniqueName: \"kubernetes.io/projected/8391482c-3c1a-4ad3-a7aa-c3553a64989d-kube-api-access-mj525\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.255506 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.255419 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-workload-socket\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.255506 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.255503 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-podinfo\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.255691 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.255571 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-token\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.255691 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.255597 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-data\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.255691 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.255618 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-workload-certs\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.255691 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.255640 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istiod-ca-cert\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.356183 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.356099 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-token\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.356183 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.356133 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-data\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.356183 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.356154 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-workload-certs\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.356460 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.356255 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istiod-ca-cert\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.356460 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.356299 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-envoy\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.356460 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.356377 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-credential-socket\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.356460 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.356406 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj525\" (UniqueName: \"kubernetes.io/projected/8391482c-3c1a-4ad3-a7aa-c3553a64989d-kube-api-access-mj525\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.356460 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.356444 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-workload-socket\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.356754 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.356576 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-workload-certs\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.356754 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.356614 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-data\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.356754 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.356671 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-podinfo\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.356754 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.356730 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-credential-socket\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.356932 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.356800 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-workload-socket\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.356977 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.356963 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istiod-ca-cert\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.358541 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.358523 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-envoy\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.358940 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.358924 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-podinfo\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.364565 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.364537 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-token\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.364717 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.364698 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj525\" (UniqueName: \"kubernetes.io/projected/8391482c-3c1a-4ad3-a7aa-c3553a64989d-kube-api-access-mj525\") pod \"e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.407792 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.407759 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:42.547395 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.547367 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s"] Jul 22 02:12:42.549403 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:12:42.549371 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8391482c_3c1a_4ad3_a7aa_c3553a64989d.slice/crio-17eff79cad32cfd1f52a9d75af45983787f869ee6408cdd40fffa5b02456f3fc WatchSource:0}: Error finding container 17eff79cad32cfd1f52a9d75af45983787f869ee6408cdd40fffa5b02456f3fc: Status 404 returned error can't find the container with id 17eff79cad32cfd1f52a9d75af45983787f869ee6408cdd40fffa5b02456f3fc Jul 22 02:12:42.551475 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.551440 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:12:42.551575 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.551507 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:12:42.551575 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.551537 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:12:42.875638 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.875543 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" event={"ID":"8391482c-3c1a-4ad3-a7aa-c3553a64989d","Type":"ContainerStarted","Data":"a31543235285d89de22dbac5522d105450052a0a76a80f3414c1c8f1502340e7"} Jul 22 02:12:42.875638 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.875610 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" event={"ID":"8391482c-3c1a-4ad3-a7aa-c3553a64989d","Type":"ContainerStarted","Data":"17eff79cad32cfd1f52a9d75af45983787f869ee6408cdd40fffa5b02456f3fc"} Jul 22 02:12:42.897233 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:42.897177 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" podStartSLOduration=0.897160649 podStartE2EDuration="897.160649ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:12:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:12:42.895752037 +0000 UTC m=+2147.725000384" watchObservedRunningTime="2026-07-22 02:12:42.897160649 +0000 UTC m=+2147.726408996" Jul 22 02:12:43.408461 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:43.408422 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:43.413386 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:43.413336 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:43.880733 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:43.880704 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:43.881559 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:43.881539 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:12:45.062899 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.062857 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh"] Jul 22 02:12:45.063535 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.063512 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="311b2b28-94f0-4ce9-b7da-0f306ec258c4" containerName="istio-proxy" Jul 22 02:12:45.066915 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.066894 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" Jul 22 02:12:45.069104 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.069084 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-b0ac34-serving-cert\"" Jul 22 02:12:45.069375 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.069355 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 02:12:45.069459 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.069424 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-49lpj\"" Jul 22 02:12:45.070112 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.070089 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 22 02:12:45.078747 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.078724 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh"] Jul 22 02:12:45.186497 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.186457 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c86mt\" (UniqueName: \"kubernetes.io/projected/47c653f6-4920-4bbb-addc-9132674c8aff-kube-api-access-c86mt\") pod \"maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh\" (UID: \"47c653f6-4920-4bbb-addc-9132674c8aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" Jul 22 02:12:45.186692 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.186570 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47c653f6-4920-4bbb-addc-9132674c8aff-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh\" (UID: \"47c653f6-4920-4bbb-addc-9132674c8aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" Jul 22 02:12:45.186692 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.186613 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47c653f6-4920-4bbb-addc-9132674c8aff-maas-api-tls\") pod \"maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh\" (UID: \"47c653f6-4920-4bbb-addc-9132674c8aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" Jul 22 02:12:45.287406 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.287329 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c86mt\" (UniqueName: \"kubernetes.io/projected/47c653f6-4920-4bbb-addc-9132674c8aff-kube-api-access-c86mt\") pod \"maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh\" (UID: \"47c653f6-4920-4bbb-addc-9132674c8aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" Jul 22 02:12:45.287608 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.287471 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47c653f6-4920-4bbb-addc-9132674c8aff-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh\" (UID: \"47c653f6-4920-4bbb-addc-9132674c8aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" Jul 22 02:12:45.287608 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.287529 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47c653f6-4920-4bbb-addc-9132674c8aff-maas-api-tls\") pod \"maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh\" (UID: \"47c653f6-4920-4bbb-addc-9132674c8aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" Jul 22 02:12:45.288159 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.288133 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47c653f6-4920-4bbb-addc-9132674c8aff-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh\" (UID: \"47c653f6-4920-4bbb-addc-9132674c8aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" Jul 22 02:12:45.289972 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.289951 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47c653f6-4920-4bbb-addc-9132674c8aff-maas-api-tls\") pod \"maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh\" (UID: \"47c653f6-4920-4bbb-addc-9132674c8aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" Jul 22 02:12:45.295882 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.295853 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c86mt\" (UniqueName: \"kubernetes.io/projected/47c653f6-4920-4bbb-addc-9132674c8aff-kube-api-access-c86mt\") pod \"maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh\" (UID: \"47c653f6-4920-4bbb-addc-9132674c8aff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" Jul 22 02:12:45.378543 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.378512 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" Jul 22 02:12:45.526972 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.526946 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh"] Jul 22 02:12:45.528536 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:12:45.528507 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47c653f6_4920_4bbb_addc_9132674c8aff.slice/crio-e4cf88bff70a52dc33fc87d339cfdd17e5a02acbb81098013c153b2689197e84 WatchSource:0}: Error finding container e4cf88bff70a52dc33fc87d339cfdd17e5a02acbb81098013c153b2689197e84: Status 404 returned error can't find the container with id e4cf88bff70a52dc33fc87d339cfdd17e5a02acbb81098013c153b2689197e84 Jul 22 02:12:45.530330 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.530312 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:12:45.891512 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:45.891476 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" event={"ID":"47c653f6-4920-4bbb-addc-9132674c8aff","Type":"ContainerStarted","Data":"e4cf88bff70a52dc33fc87d339cfdd17e5a02acbb81098013c153b2689197e84"} Jul 22 02:12:46.898186 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:46.898146 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" event={"ID":"47c653f6-4920-4bbb-addc-9132674c8aff","Type":"ContainerStarted","Data":"ac5d8613a862053cbc9bfbe74ad96b04d70ce60e6dcda53dca74aefac251aa21"} Jul 22 02:12:46.898643 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:46.898300 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" Jul 22 02:12:46.918076 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:46.918025 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" podStartSLOduration=1.494278278 podStartE2EDuration="1.918010738s" totalImagesPullingTime="423.73246ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:12:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:12:46.916264775 +0000 UTC m=+2151.745513134" watchObservedRunningTime="2026-07-22 02:12:46.918010738 +0000 UTC m=+2151.747259085" Jul 22 02:12:52.908365 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:52.908311 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" Jul 22 02:12:53.183872 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:53.183790 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv"] Jul 22 02:12:53.187587 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:53.187566 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" Jul 22 02:12:53.191936 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:53.191911 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-1ad649-serving-cert\"" Jul 22 02:12:53.198723 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:53.198693 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv"] Jul 22 02:12:53.266449 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:53.266408 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnp4r\" (UniqueName: \"kubernetes.io/projected/5a2539b4-536e-4986-b893-dda9aae4d5bb-kube-api-access-wnp4r\") pod \"maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv\" (UID: \"5a2539b4-536e-4986-b893-dda9aae4d5bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" Jul 22 02:12:53.266634 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:53.266480 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a2539b4-536e-4986-b893-dda9aae4d5bb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv\" (UID: \"5a2539b4-536e-4986-b893-dda9aae4d5bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" Jul 22 02:12:53.266634 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:53.266620 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a2539b4-536e-4986-b893-dda9aae4d5bb-maas-api-tls\") pod \"maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv\" (UID: \"5a2539b4-536e-4986-b893-dda9aae4d5bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" Jul 22 02:12:53.367863 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:53.367822 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a2539b4-536e-4986-b893-dda9aae4d5bb-maas-api-tls\") pod \"maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv\" (UID: \"5a2539b4-536e-4986-b893-dda9aae4d5bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" Jul 22 02:12:53.368051 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:53.367890 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnp4r\" (UniqueName: \"kubernetes.io/projected/5a2539b4-536e-4986-b893-dda9aae4d5bb-kube-api-access-wnp4r\") pod \"maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv\" (UID: \"5a2539b4-536e-4986-b893-dda9aae4d5bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" Jul 22 02:12:53.368051 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:53.367952 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a2539b4-536e-4986-b893-dda9aae4d5bb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv\" (UID: \"5a2539b4-536e-4986-b893-dda9aae4d5bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" Jul 22 02:12:53.368700 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:53.368677 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a2539b4-536e-4986-b893-dda9aae4d5bb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv\" (UID: \"5a2539b4-536e-4986-b893-dda9aae4d5bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" Jul 22 02:12:53.370396 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:53.370373 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a2539b4-536e-4986-b893-dda9aae4d5bb-maas-api-tls\") pod \"maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv\" (UID: \"5a2539b4-536e-4986-b893-dda9aae4d5bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" Jul 22 02:12:53.391780 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:53.391755 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnp4r\" (UniqueName: \"kubernetes.io/projected/5a2539b4-536e-4986-b893-dda9aae4d5bb-kube-api-access-wnp4r\") pod \"maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv\" (UID: \"5a2539b4-536e-4986-b893-dda9aae4d5bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" Jul 22 02:12:53.501151 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:53.501045 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" Jul 22 02:12:53.704477 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:53.704449 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv"] Jul 22 02:12:53.707377 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:12:53.707332 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a2539b4_536e_4986_b893_dda9aae4d5bb.slice/crio-2cdf4e649a03b245bcca15af09d5c48f465f87d9437f104436e71889d2f67e57 WatchSource:0}: Error finding container 2cdf4e649a03b245bcca15af09d5c48f465f87d9437f104436e71889d2f67e57: Status 404 returned error can't find the container with id 2cdf4e649a03b245bcca15af09d5c48f465f87d9437f104436e71889d2f67e57 Jul 22 02:12:53.944227 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:53.944141 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" event={"ID":"5a2539b4-536e-4986-b893-dda9aae4d5bb","Type":"ContainerStarted","Data":"2cdf4e649a03b245bcca15af09d5c48f465f87d9437f104436e71889d2f67e57"} Jul 22 02:12:54.954111 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:54.954066 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" event={"ID":"5a2539b4-536e-4986-b893-dda9aae4d5bb","Type":"ContainerStarted","Data":"f451628f35e41d59ce25ef481734221bca3246da4a38f13d8b529fe1f2d0ecc8"} Jul 22 02:12:54.955023 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:54.954994 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" Jul 22 02:12:54.975451 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:54.975396 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" podStartSLOduration=1.633539055 podStartE2EDuration="1.975378394s" totalImagesPullingTime="341.839339ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:12:54.973301354 +0000 UTC m=+2159.802549694" watchObservedRunningTime="2026-07-22 02:12:54.975378394 +0000 UTC m=+2159.804626774" Jul 22 02:12:58.882637 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:58.882550 2559 patch_prober.go:28] interesting pod/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 22 02:12:58.882637 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:58.882620 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" podUID="8391482c-3c1a-4ad3-a7aa-c3553a64989d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 22 02:12:58.976834 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:58.976797 2559 generic.go:356] "Generic (PLEG): container finished" podID="8391482c-3c1a-4ad3-a7aa-c3553a64989d" containerID="a31543235285d89de22dbac5522d105450052a0a76a80f3414c1c8f1502340e7" exitCode=0 Jul 22 02:12:58.977006 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:58.976872 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" event={"ID":"8391482c-3c1a-4ad3-a7aa-c3553a64989d","Type":"ContainerDied","Data":"a31543235285d89de22dbac5522d105450052a0a76a80f3414c1c8f1502340e7"} Jul 22 02:12:58.977300 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:58.977285 2559 scope.go:117] "RemoveContainer" containerID="a31543235285d89de22dbac5522d105450052a0a76a80f3414c1c8f1502340e7" Jul 22 02:12:58.978369 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:58.978326 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:12:58.978434 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:58.978416 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:12:58.978476 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:58.978454 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:12:59.986000 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:12:59.985968 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" event={"ID":"8391482c-3c1a-4ad3-a7aa-c3553a64989d","Type":"ContainerStarted","Data":"bb1324a16b757e27f2178170708d2cd6988a3b94594d32ca7cc21cd3491a9e9f"} Jul 22 02:13:00.408868 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:13:00.408825 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:13:00.409143 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:13:00.409118 2559 patch_prober.go:28] interesting pod/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 22 02:13:00.409191 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:13:00.409169 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" podUID="8391482c-3c1a-4ad3-a7aa-c3553a64989d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 22 02:13:01.409188 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:13:01.409136 2559 patch_prober.go:28] interesting pod/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 22 02:13:01.409654 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:13:01.409203 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" podUID="8391482c-3c1a-4ad3-a7aa-c3553a64989d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 22 02:13:01.970938 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:13:01.970909 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" Jul 22 02:13:02.408774 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:13:02.408732 2559 patch_prober.go:28] interesting pod/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 22 02:13:02.408994 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:13:02.408799 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" podUID="8391482c-3c1a-4ad3-a7aa-c3553a64989d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 22 02:13:03.408303 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:13:03.408260 2559 patch_prober.go:28] interesting pod/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 22 02:13:03.408727 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:13:03.408320 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" podUID="8391482c-3c1a-4ad3-a7aa-c3553a64989d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 22 02:13:04.412162 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:13:04.412133 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:13:04.412572 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:13:04.412545 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:13:04.413116 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:13:04.413094 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:14:40.023011 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.022978 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh"] Jul 22 02:14:40.023600 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.023196 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" podUID="47c653f6-4920-4bbb-addc-9132674c8aff" containerName="maas-api" containerID="cri-o://ac5d8613a862053cbc9bfbe74ad96b04d70ce60e6dcda53dca74aefac251aa21" gracePeriod=30 Jul 22 02:14:40.283455 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.283387 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" Jul 22 02:14:40.341506 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.341468 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c86mt\" (UniqueName: \"kubernetes.io/projected/47c653f6-4920-4bbb-addc-9132674c8aff-kube-api-access-c86mt\") pod \"47c653f6-4920-4bbb-addc-9132674c8aff\" (UID: \"47c653f6-4920-4bbb-addc-9132674c8aff\") " Jul 22 02:14:40.341699 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.341589 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47c653f6-4920-4bbb-addc-9132674c8aff-maas-api-tls\") pod \"47c653f6-4920-4bbb-addc-9132674c8aff\" (UID: \"47c653f6-4920-4bbb-addc-9132674c8aff\") " Jul 22 02:14:40.341699 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.341660 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47c653f6-4920-4bbb-addc-9132674c8aff-openshift-service-ca\") pod \"47c653f6-4920-4bbb-addc-9132674c8aff\" (UID: \"47c653f6-4920-4bbb-addc-9132674c8aff\") " Jul 22 02:14:40.342026 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.341993 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47c653f6-4920-4bbb-addc-9132674c8aff-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "47c653f6-4920-4bbb-addc-9132674c8aff" (UID: "47c653f6-4920-4bbb-addc-9132674c8aff"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:14:40.343571 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.343546 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47c653f6-4920-4bbb-addc-9132674c8aff-kube-api-access-c86mt" (OuterVolumeSpecName: "kube-api-access-c86mt") pod "47c653f6-4920-4bbb-addc-9132674c8aff" (UID: "47c653f6-4920-4bbb-addc-9132674c8aff"). InnerVolumeSpecName "kube-api-access-c86mt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:14:40.343730 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.343633 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47c653f6-4920-4bbb-addc-9132674c8aff-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "47c653f6-4920-4bbb-addc-9132674c8aff" (UID: "47c653f6-4920-4bbb-addc-9132674c8aff"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:14:40.443183 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.443140 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/47c653f6-4920-4bbb-addc-9132674c8aff-maas-api-tls\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:14:40.443183 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.443175 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/47c653f6-4920-4bbb-addc-9132674c8aff-openshift-service-ca\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:14:40.443183 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.443185 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c86mt\" (UniqueName: \"kubernetes.io/projected/47c653f6-4920-4bbb-addc-9132674c8aff-kube-api-access-c86mt\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:14:40.484295 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.484265 2559 generic.go:356] "Generic (PLEG): container finished" podID="47c653f6-4920-4bbb-addc-9132674c8aff" containerID="ac5d8613a862053cbc9bfbe74ad96b04d70ce60e6dcda53dca74aefac251aa21" exitCode=0 Jul 22 02:14:40.484496 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.484316 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" Jul 22 02:14:40.484496 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.484319 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" event={"ID":"47c653f6-4920-4bbb-addc-9132674c8aff","Type":"ContainerDied","Data":"ac5d8613a862053cbc9bfbe74ad96b04d70ce60e6dcda53dca74aefac251aa21"} Jul 22 02:14:40.484496 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.484397 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh" event={"ID":"47c653f6-4920-4bbb-addc-9132674c8aff","Type":"ContainerDied","Data":"e4cf88bff70a52dc33fc87d339cfdd17e5a02acbb81098013c153b2689197e84"} Jul 22 02:14:40.484496 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.484418 2559 scope.go:117] "RemoveContainer" containerID="ac5d8613a862053cbc9bfbe74ad96b04d70ce60e6dcda53dca74aefac251aa21" Jul 22 02:14:40.493223 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.493197 2559 scope.go:117] "RemoveContainer" containerID="ac5d8613a862053cbc9bfbe74ad96b04d70ce60e6dcda53dca74aefac251aa21" Jul 22 02:14:40.493496 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:14:40.493478 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac5d8613a862053cbc9bfbe74ad96b04d70ce60e6dcda53dca74aefac251aa21\": container with ID starting with ac5d8613a862053cbc9bfbe74ad96b04d70ce60e6dcda53dca74aefac251aa21 not found: ID does not exist" containerID="ac5d8613a862053cbc9bfbe74ad96b04d70ce60e6dcda53dca74aefac251aa21" Jul 22 02:14:40.493556 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.493507 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac5d8613a862053cbc9bfbe74ad96b04d70ce60e6dcda53dca74aefac251aa21"} err="failed to get container status \"ac5d8613a862053cbc9bfbe74ad96b04d70ce60e6dcda53dca74aefac251aa21\": rpc error: code = NotFound desc = could not find container \"ac5d8613a862053cbc9bfbe74ad96b04d70ce60e6dcda53dca74aefac251aa21\": container with ID starting with ac5d8613a862053cbc9bfbe74ad96b04d70ce60e6dcda53dca74aefac251aa21 not found: ID does not exist" Jul 22 02:14:40.512571 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.512536 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh"] Jul 22 02:14:40.521145 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:40.521109 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b0ac34-6596dfc9d5-wccsh"] Jul 22 02:14:41.626327 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:41.626293 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47c653f6-4920-4bbb-addc-9132674c8aff" path="/var/lib/kubelet/pods/47c653f6-4920-4bbb-addc-9132674c8aff/volumes" Jul 22 02:14:51.756231 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:51.756191 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s"] Jul 22 02:14:51.756669 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:51.756441 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" podUID="8391482c-3c1a-4ad3-a7aa-c3553a64989d" containerName="istio-proxy" containerID="cri-o://bb1324a16b757e27f2178170708d2cd6988a3b94594d32ca7cc21cd3491a9e9f" gracePeriod=30 Jul 22 02:14:57.018443 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.018372 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:14:57.119329 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.119280 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-workload-socket\") pod \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " Jul 22 02:14:57.119567 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.119396 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mj525\" (UniqueName: \"kubernetes.io/projected/8391482c-3c1a-4ad3-a7aa-c3553a64989d-kube-api-access-mj525\") pod \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " Jul 22 02:14:57.119567 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.119462 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-podinfo\") pod \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " Jul 22 02:14:57.119567 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.119483 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istiod-ca-cert\") pod \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " Jul 22 02:14:57.119567 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.119540 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-envoy\") pod \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " Jul 22 02:14:57.119796 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.119573 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-data\") pod \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " Jul 22 02:14:57.119796 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.119605 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-workload-certs\") pod \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " Jul 22 02:14:57.119796 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.119615 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8391482c-3c1a-4ad3-a7aa-c3553a64989d" (UID: "8391482c-3c1a-4ad3-a7aa-c3553a64989d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:14:57.119796 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.119640 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-token\") pod \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " Jul 22 02:14:57.119796 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.119691 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-credential-socket\") pod \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\" (UID: \"8391482c-3c1a-4ad3-a7aa-c3553a64989d\") " Jul 22 02:14:57.120026 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.119938 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8391482c-3c1a-4ad3-a7aa-c3553a64989d" (UID: "8391482c-3c1a-4ad3-a7aa-c3553a64989d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:14:57.120026 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.119939 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8391482c-3c1a-4ad3-a7aa-c3553a64989d" (UID: "8391482c-3c1a-4ad3-a7aa-c3553a64989d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:14:57.120143 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.120056 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-workload-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:14:57.120143 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.120074 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istiod-ca-cert\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:14:57.120143 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.120087 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-workload-certs\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:14:57.120302 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.120186 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8391482c-3c1a-4ad3-a7aa-c3553a64989d" (UID: "8391482c-3c1a-4ad3-a7aa-c3553a64989d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:14:57.120938 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.120910 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-data" (OuterVolumeSpecName: "istio-data") pod "8391482c-3c1a-4ad3-a7aa-c3553a64989d" (UID: "8391482c-3c1a-4ad3-a7aa-c3553a64989d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:14:57.121761 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.121738 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-token" (OuterVolumeSpecName: "istio-token") pod "8391482c-3c1a-4ad3-a7aa-c3553a64989d" (UID: "8391482c-3c1a-4ad3-a7aa-c3553a64989d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:14:57.121851 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.121800 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8391482c-3c1a-4ad3-a7aa-c3553a64989d" (UID: "8391482c-3c1a-4ad3-a7aa-c3553a64989d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:14:57.121917 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.121887 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8391482c-3c1a-4ad3-a7aa-c3553a64989d" (UID: "8391482c-3c1a-4ad3-a7aa-c3553a64989d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:14:57.121984 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.121965 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8391482c-3c1a-4ad3-a7aa-c3553a64989d-kube-api-access-mj525" (OuterVolumeSpecName: "kube-api-access-mj525") pod "8391482c-3c1a-4ad3-a7aa-c3553a64989d" (UID: "8391482c-3c1a-4ad3-a7aa-c3553a64989d"). InnerVolumeSpecName "kube-api-access-mj525". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:14:57.220667 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.220632 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mj525\" (UniqueName: \"kubernetes.io/projected/8391482c-3c1a-4ad3-a7aa-c3553a64989d-kube-api-access-mj525\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:14:57.220667 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.220663 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-podinfo\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:14:57.220667 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.220672 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-envoy\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:14:57.220965 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.220680 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-data\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:14:57.220965 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.220698 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8391482c-3c1a-4ad3-a7aa-c3553a64989d-istio-token\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:14:57.220965 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.220708 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8391482c-3c1a-4ad3-a7aa-c3553a64989d-credential-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:14:57.569623 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.569585 2559 generic.go:356] "Generic (PLEG): container finished" podID="8391482c-3c1a-4ad3-a7aa-c3553a64989d" containerID="bb1324a16b757e27f2178170708d2cd6988a3b94594d32ca7cc21cd3491a9e9f" exitCode=0 Jul 22 02:14:57.569797 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.569629 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" Jul 22 02:14:57.569797 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.569675 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" event={"ID":"8391482c-3c1a-4ad3-a7aa-c3553a64989d","Type":"ContainerDied","Data":"bb1324a16b757e27f2178170708d2cd6988a3b94594d32ca7cc21cd3491a9e9f"} Jul 22 02:14:57.569797 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.569732 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s" event={"ID":"8391482c-3c1a-4ad3-a7aa-c3553a64989d","Type":"ContainerDied","Data":"17eff79cad32cfd1f52a9d75af45983787f869ee6408cdd40fffa5b02456f3fc"} Jul 22 02:14:57.569797 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.569755 2559 scope.go:117] "RemoveContainer" containerID="bb1324a16b757e27f2178170708d2cd6988a3b94594d32ca7cc21cd3491a9e9f" Jul 22 02:14:57.579245 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.579226 2559 scope.go:117] "RemoveContainer" containerID="a31543235285d89de22dbac5522d105450052a0a76a80f3414c1c8f1502340e7" Jul 22 02:14:57.587158 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.587138 2559 scope.go:117] "RemoveContainer" containerID="bb1324a16b757e27f2178170708d2cd6988a3b94594d32ca7cc21cd3491a9e9f" Jul 22 02:14:57.587503 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:14:57.587480 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb1324a16b757e27f2178170708d2cd6988a3b94594d32ca7cc21cd3491a9e9f\": container with ID starting with bb1324a16b757e27f2178170708d2cd6988a3b94594d32ca7cc21cd3491a9e9f not found: ID does not exist" containerID="bb1324a16b757e27f2178170708d2cd6988a3b94594d32ca7cc21cd3491a9e9f" Jul 22 02:14:57.587603 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.587516 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb1324a16b757e27f2178170708d2cd6988a3b94594d32ca7cc21cd3491a9e9f"} err="failed to get container status \"bb1324a16b757e27f2178170708d2cd6988a3b94594d32ca7cc21cd3491a9e9f\": rpc error: code = NotFound desc = could not find container \"bb1324a16b757e27f2178170708d2cd6988a3b94594d32ca7cc21cd3491a9e9f\": container with ID starting with bb1324a16b757e27f2178170708d2cd6988a3b94594d32ca7cc21cd3491a9e9f not found: ID does not exist" Jul 22 02:14:57.587603 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.587542 2559 scope.go:117] "RemoveContainer" containerID="a31543235285d89de22dbac5522d105450052a0a76a80f3414c1c8f1502340e7" Jul 22 02:14:57.587814 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:14:57.587795 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a31543235285d89de22dbac5522d105450052a0a76a80f3414c1c8f1502340e7\": container with ID starting with a31543235285d89de22dbac5522d105450052a0a76a80f3414c1c8f1502340e7 not found: ID does not exist" containerID="a31543235285d89de22dbac5522d105450052a0a76a80f3414c1c8f1502340e7" Jul 22 02:14:57.587859 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.587821 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a31543235285d89de22dbac5522d105450052a0a76a80f3414c1c8f1502340e7"} err="failed to get container status \"a31543235285d89de22dbac5522d105450052a0a76a80f3414c1c8f1502340e7\": rpc error: code = NotFound desc = could not find container \"a31543235285d89de22dbac5522d105450052a0a76a80f3414c1c8f1502340e7\": container with ID starting with a31543235285d89de22dbac5522d105450052a0a76a80f3414c1c8f1502340e7 not found: ID does not exist" Jul 22 02:14:57.596474 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.596451 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s"] Jul 22 02:14:57.602983 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.602961 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-b0ac34-openshift-default-8c58bc667-ghs7s"] Jul 22 02:14:57.626208 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:14:57.626181 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8391482c-3c1a-4ad3-a7aa-c3553a64989d" path="/var/lib/kubelet/pods/8391482c-3c1a-4ad3-a7aa-c3553a64989d/volumes" Jul 22 02:15:00.157529 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:00.157495 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1ad649-29744775-l66sw"] Jul 22 02:15:00.157927 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:00.157866 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8391482c-3c1a-4ad3-a7aa-c3553a64989d" containerName="istio-proxy" Jul 22 02:15:00.157927 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:00.157877 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8391482c-3c1a-4ad3-a7aa-c3553a64989d" containerName="istio-proxy" Jul 22 02:15:00.157927 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:00.157887 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47c653f6-4920-4bbb-addc-9132674c8aff" containerName="maas-api" Jul 22 02:15:00.157927 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:00.157893 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="47c653f6-4920-4bbb-addc-9132674c8aff" containerName="maas-api" Jul 22 02:15:00.157927 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:00.157899 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8391482c-3c1a-4ad3-a7aa-c3553a64989d" containerName="istio-proxy" Jul 22 02:15:00.157927 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:00.157904 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8391482c-3c1a-4ad3-a7aa-c3553a64989d" containerName="istio-proxy" Jul 22 02:15:00.158128 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:00.157969 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8391482c-3c1a-4ad3-a7aa-c3553a64989d" containerName="istio-proxy" Jul 22 02:15:00.158128 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:00.157978 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8391482c-3c1a-4ad3-a7aa-c3553a64989d" containerName="istio-proxy" Jul 22 02:15:00.158128 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:00.157987 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="47c653f6-4920-4bbb-addc-9132674c8aff" containerName="maas-api" Jul 22 02:15:00.162713 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:00.162681 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1ad649-29744775-l66sw" Jul 22 02:15:00.165619 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:00.165597 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cqdw8\"" Jul 22 02:15:00.171542 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:00.171517 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1ad649-29744775-l66sw"] Jul 22 02:15:00.174047 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:00.174024 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1ad649-29744775-l66sw" Jul 22 02:15:00.316461 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:00.316431 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1ad649-29744775-l66sw"] Jul 22 02:15:00.318884 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:15:00.318859 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b67359a_0ed7_4923_adc9_fbc7d1e0aca1.slice/crio-81310296f5dbdd487f86b1949287c757293dada095eb8f51303f7fef4d3c7827 WatchSource:0}: Error finding container 81310296f5dbdd487f86b1949287c757293dada095eb8f51303f7fef4d3c7827: Status 404 returned error can't find the container with id 81310296f5dbdd487f86b1949287c757293dada095eb8f51303f7fef4d3c7827 Jul 22 02:15:00.586774 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:00.586736 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1ad649-29744775-l66sw" event={"ID":"0b67359a-0ed7-4923-adc9-fbc7d1e0aca1","Type":"ContainerStarted","Data":"81310296f5dbdd487f86b1949287c757293dada095eb8f51303f7fef4d3c7827"} Jul 22 02:15:01.593783 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:01.593752 2559 generic.go:356] "Generic (PLEG): container finished" podID="0b67359a-0ed7-4923-adc9-fbc7d1e0aca1" containerID="5f132e213812896d421ad4713901f2fe5807ba4dd053e8cb630fabf8d9bc6a21" exitCode=0 Jul 22 02:15:01.594182 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:01.593800 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1ad649-29744775-l66sw" event={"ID":"0b67359a-0ed7-4923-adc9-fbc7d1e0aca1","Type":"ContainerDied","Data":"5f132e213812896d421ad4713901f2fe5807ba4dd053e8cb630fabf8d9bc6a21"} Jul 22 02:15:02.735311 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:02.735272 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1ad649-29744775-l66sw" Jul 22 02:15:02.839979 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:02.839945 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv"] Jul 22 02:15:02.840222 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:02.840183 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" podUID="5a2539b4-536e-4986-b893-dda9aae4d5bb" containerName="maas-api" containerID="cri-o://f451628f35e41d59ce25ef481734221bca3246da4a38f13d8b529fe1f2d0ecc8" gracePeriod=30 Jul 22 02:15:02.890413 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:02.890378 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1ad649-29744775-l66sw"] Jul 22 02:15:02.897989 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:02.897953 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1ad649-29744775-l66sw"] Jul 22 02:15:03.111921 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.111850 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" Jul 22 02:15:03.282947 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.282906 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnp4r\" (UniqueName: \"kubernetes.io/projected/5a2539b4-536e-4986-b893-dda9aae4d5bb-kube-api-access-wnp4r\") pod \"5a2539b4-536e-4986-b893-dda9aae4d5bb\" (UID: \"5a2539b4-536e-4986-b893-dda9aae4d5bb\") " Jul 22 02:15:03.283150 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.282988 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a2539b4-536e-4986-b893-dda9aae4d5bb-maas-api-tls\") pod \"5a2539b4-536e-4986-b893-dda9aae4d5bb\" (UID: \"5a2539b4-536e-4986-b893-dda9aae4d5bb\") " Jul 22 02:15:03.283150 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.283118 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a2539b4-536e-4986-b893-dda9aae4d5bb-openshift-service-ca\") pod \"5a2539b4-536e-4986-b893-dda9aae4d5bb\" (UID: \"5a2539b4-536e-4986-b893-dda9aae4d5bb\") " Jul 22 02:15:03.283536 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.283497 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a2539b4-536e-4986-b893-dda9aae4d5bb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5a2539b4-536e-4986-b893-dda9aae4d5bb" (UID: "5a2539b4-536e-4986-b893-dda9aae4d5bb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:15:03.285067 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.285048 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a2539b4-536e-4986-b893-dda9aae4d5bb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5a2539b4-536e-4986-b893-dda9aae4d5bb" (UID: "5a2539b4-536e-4986-b893-dda9aae4d5bb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:15:03.285422 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.285393 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a2539b4-536e-4986-b893-dda9aae4d5bb-kube-api-access-wnp4r" (OuterVolumeSpecName: "kube-api-access-wnp4r") pod "5a2539b4-536e-4986-b893-dda9aae4d5bb" (UID: "5a2539b4-536e-4986-b893-dda9aae4d5bb"). InnerVolumeSpecName "kube-api-access-wnp4r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:15:03.384904 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.384799 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnp4r\" (UniqueName: \"kubernetes.io/projected/5a2539b4-536e-4986-b893-dda9aae4d5bb-kube-api-access-wnp4r\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:15:03.384904 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.384847 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a2539b4-536e-4986-b893-dda9aae4d5bb-maas-api-tls\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:15:03.384904 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.384857 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a2539b4-536e-4986-b893-dda9aae4d5bb-openshift-service-ca\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:15:03.605157 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.605124 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1ad649-29744775-l66sw" Jul 22 02:15:03.605157 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.605150 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81310296f5dbdd487f86b1949287c757293dada095eb8f51303f7fef4d3c7827" Jul 22 02:15:03.606516 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.606483 2559 generic.go:356] "Generic (PLEG): container finished" podID="5a2539b4-536e-4986-b893-dda9aae4d5bb" containerID="f451628f35e41d59ce25ef481734221bca3246da4a38f13d8b529fe1f2d0ecc8" exitCode=0 Jul 22 02:15:03.606674 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.606526 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" Jul 22 02:15:03.606674 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.606526 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" event={"ID":"5a2539b4-536e-4986-b893-dda9aae4d5bb","Type":"ContainerDied","Data":"f451628f35e41d59ce25ef481734221bca3246da4a38f13d8b529fe1f2d0ecc8"} Jul 22 02:15:03.606674 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.606661 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv" event={"ID":"5a2539b4-536e-4986-b893-dda9aae4d5bb","Type":"ContainerDied","Data":"2cdf4e649a03b245bcca15af09d5c48f465f87d9437f104436e71889d2f67e57"} Jul 22 02:15:03.606865 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.606683 2559 scope.go:117] "RemoveContainer" containerID="f451628f35e41d59ce25ef481734221bca3246da4a38f13d8b529fe1f2d0ecc8" Jul 22 02:15:03.616276 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.616253 2559 scope.go:117] "RemoveContainer" containerID="f451628f35e41d59ce25ef481734221bca3246da4a38f13d8b529fe1f2d0ecc8" Jul 22 02:15:03.616644 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:15:03.616622 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f451628f35e41d59ce25ef481734221bca3246da4a38f13d8b529fe1f2d0ecc8\": container with ID starting with f451628f35e41d59ce25ef481734221bca3246da4a38f13d8b529fe1f2d0ecc8 not found: ID does not exist" containerID="f451628f35e41d59ce25ef481734221bca3246da4a38f13d8b529fe1f2d0ecc8" Jul 22 02:15:03.616738 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.616656 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f451628f35e41d59ce25ef481734221bca3246da4a38f13d8b529fe1f2d0ecc8"} err="failed to get container status \"f451628f35e41d59ce25ef481734221bca3246da4a38f13d8b529fe1f2d0ecc8\": rpc error: code = NotFound desc = could not find container \"f451628f35e41d59ce25ef481734221bca3246da4a38f13d8b529fe1f2d0ecc8\": container with ID starting with f451628f35e41d59ce25ef481734221bca3246da4a38f13d8b529fe1f2d0ecc8 not found: ID does not exist" Jul 22 02:15:03.625678 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.625650 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b67359a-0ed7-4923-adc9-fbc7d1e0aca1" path="/var/lib/kubelet/pods/0b67359a-0ed7-4923-adc9-fbc7d1e0aca1/volumes" Jul 22 02:15:03.636006 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.635941 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv"] Jul 22 02:15:03.639632 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:03.639611 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1ad649-7b9c79f688-rtmsv"] Jul 22 02:15:03.640413 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:15:03.640385 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-key-cleanup-e2e-inf-b-1ad649-29744775-l66sw\" is forbidden: User \"system:node:ip-10-0-131-80.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-131-80.ec2.internal' and this object" podUID="0b67359a-0ed7-4923-adc9-fbc7d1e0aca1" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1ad649-29744775-l66sw" Jul 22 02:15:05.627603 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:05.627568 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a2539b4-536e-4986-b893-dda9aae4d5bb" path="/var/lib/kubelet/pods/5a2539b4-536e-4986-b893-dda9aae4d5bb/volumes" Jul 22 02:15:15.382518 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.382483 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x"] Jul 22 02:15:15.382994 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.382893 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a2539b4-536e-4986-b893-dda9aae4d5bb" containerName="maas-api" Jul 22 02:15:15.382994 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.382908 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a2539b4-536e-4986-b893-dda9aae4d5bb" containerName="maas-api" Jul 22 02:15:15.382994 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.382920 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b67359a-0ed7-4923-adc9-fbc7d1e0aca1" containerName="cleanup" Jul 22 02:15:15.382994 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.382925 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b67359a-0ed7-4923-adc9-fbc7d1e0aca1" containerName="cleanup" Jul 22 02:15:15.382994 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.382991 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a2539b4-536e-4986-b893-dda9aae4d5bb" containerName="maas-api" Jul 22 02:15:15.383157 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.383001 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b67359a-0ed7-4923-adc9-fbc7d1e0aca1" containerName="cleanup" Jul 22 02:15:15.387825 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.387807 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.390770 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.390745 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ae6764-openshift-default-dockercfg-kfxl8\"" Jul 22 02:15:15.399953 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.399925 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x"] Jul 22 02:15:15.501647 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.501601 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-data\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.501850 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.501670 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-envoy\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.501850 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.501701 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-podinfo\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.501850 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.501734 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-workload-socket\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.501850 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.501799 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mrrb\" (UniqueName: \"kubernetes.io/projected/2f341a5b-981d-4adf-aab2-353d5c890d7d-kube-api-access-4mrrb\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.501850 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.501828 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-token\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.502086 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.501860 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-credential-socket\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.502086 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.501892 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f341a5b-981d-4adf-aab2-353d5c890d7d-istiod-ca-cert\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.502086 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.501925 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-workload-certs\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.602698 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.602661 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-data\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.602698 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.602705 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-envoy\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.602971 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.602801 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-podinfo\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.602971 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.602846 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-workload-socket\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.602971 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.602926 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mrrb\" (UniqueName: \"kubernetes.io/projected/2f341a5b-981d-4adf-aab2-353d5c890d7d-kube-api-access-4mrrb\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.602971 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.602954 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-token\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.603186 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.602989 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-credential-socket\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.603186 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.603026 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f341a5b-981d-4adf-aab2-353d5c890d7d-istiod-ca-cert\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.603186 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.603161 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-data\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.603335 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.603217 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-workload-socket\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.603335 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.603260 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-workload-certs\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.603549 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.603517 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-credential-socket\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.603669 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.603588 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-workload-certs\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.603821 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.603799 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f341a5b-981d-4adf-aab2-353d5c890d7d-istiod-ca-cert\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.605026 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.605008 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-envoy\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.605415 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.605392 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-podinfo\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.614550 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.614529 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mrrb\" (UniqueName: \"kubernetes.io/projected/2f341a5b-981d-4adf-aab2-353d5c890d7d-kube-api-access-4mrrb\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.615081 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.615063 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-token\") pod \"e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.699015 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.698917 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:15.850944 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.850909 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x"] Jul 22 02:15:15.851201 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:15:15.851178 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f341a5b_981d_4adf_aab2_353d5c890d7d.slice/crio-8a336dedc0f1259c86958a913c9f658b252563b2aa1836506898aa150ef7096b WatchSource:0}: Error finding container 8a336dedc0f1259c86958a913c9f658b252563b2aa1836506898aa150ef7096b: Status 404 returned error can't find the container with id 8a336dedc0f1259c86958a913c9f658b252563b2aa1836506898aa150ef7096b Jul 22 02:15:15.853291 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.853259 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:15:15.853384 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.853322 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:15:15.853384 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:15.853379 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:15:16.671920 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:16.671883 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" event={"ID":"2f341a5b-981d-4adf-aab2-353d5c890d7d","Type":"ContainerStarted","Data":"4a4760eaef0f7994b85941b04141d3e9013a20ae9aa7f99260cfd5f1193d0b60"} Jul 22 02:15:16.672383 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:16.671938 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" event={"ID":"2f341a5b-981d-4adf-aab2-353d5c890d7d","Type":"ContainerStarted","Data":"8a336dedc0f1259c86958a913c9f658b252563b2aa1836506898aa150ef7096b"} Jul 22 02:15:16.699247 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:16.699185 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:16.702172 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:16.702128 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" podStartSLOduration=1.702113084 podStartE2EDuration="1.702113084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:15:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:15:16.699438156 +0000 UTC m=+2301.528686503" watchObservedRunningTime="2026-07-22 02:15:16.702113084 +0000 UTC m=+2301.531361432" Jul 22 02:15:16.704530 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:16.704513 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:17.677175 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:17.677143 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:17.678290 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:17.678263 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:18.892821 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:18.892785 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk"] Jul 22 02:15:18.896731 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:18.896707 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" Jul 22 02:15:18.902491 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:18.902469 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-ae6764-serving-cert\"" Jul 22 02:15:18.902625 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:18.902540 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 02:15:18.903402 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:18.903378 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-49lpj\"" Jul 22 02:15:18.903542 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:18.903377 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 22 02:15:18.909456 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:18.909434 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk"] Jul 22 02:15:18.937558 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:18.937526 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-898dt\" (UniqueName: \"kubernetes.io/projected/3b920554-398b-4dce-b517-4374288dc369-kube-api-access-898dt\") pod \"maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk\" (UID: \"3b920554-398b-4dce-b517-4374288dc369\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" Jul 22 02:15:18.937716 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:18.937567 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b920554-398b-4dce-b517-4374288dc369-openshift-service-ca\") pod \"maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk\" (UID: \"3b920554-398b-4dce-b517-4374288dc369\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" Jul 22 02:15:18.937716 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:18.937631 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b920554-398b-4dce-b517-4374288dc369-maas-api-tls\") pod \"maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk\" (UID: \"3b920554-398b-4dce-b517-4374288dc369\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" Jul 22 02:15:19.039103 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:19.039069 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-898dt\" (UniqueName: \"kubernetes.io/projected/3b920554-398b-4dce-b517-4374288dc369-kube-api-access-898dt\") pod \"maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk\" (UID: \"3b920554-398b-4dce-b517-4374288dc369\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" Jul 22 02:15:19.039103 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:19.039111 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b920554-398b-4dce-b517-4374288dc369-openshift-service-ca\") pod \"maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk\" (UID: \"3b920554-398b-4dce-b517-4374288dc369\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" Jul 22 02:15:19.039359 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:19.039243 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b920554-398b-4dce-b517-4374288dc369-maas-api-tls\") pod \"maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk\" (UID: \"3b920554-398b-4dce-b517-4374288dc369\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" Jul 22 02:15:19.039407 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:15:19.039385 2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-serving-cert: secret "maas-api-e2e-api-a-ae6764-serving-cert" not found Jul 22 02:15:19.039470 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:15:19.039459 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b920554-398b-4dce-b517-4374288dc369-maas-api-tls podName:3b920554-398b-4dce-b517-4374288dc369 nodeName:}" failed. No retries permitted until 2026-07-22 02:15:19.539439716 +0000 UTC m=+2304.368688051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3b920554-398b-4dce-b517-4374288dc369-maas-api-tls") pod "maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" (UID: "3b920554-398b-4dce-b517-4374288dc369") : secret "maas-api-e2e-api-a-ae6764-serving-cert" not found Jul 22 02:15:19.039750 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:19.039734 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b920554-398b-4dce-b517-4374288dc369-openshift-service-ca\") pod \"maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk\" (UID: \"3b920554-398b-4dce-b517-4374288dc369\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" Jul 22 02:15:19.051888 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:19.051858 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-898dt\" (UniqueName: \"kubernetes.io/projected/3b920554-398b-4dce-b517-4374288dc369-kube-api-access-898dt\") pod \"maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk\" (UID: \"3b920554-398b-4dce-b517-4374288dc369\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" Jul 22 02:15:19.544826 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:19.544787 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b920554-398b-4dce-b517-4374288dc369-maas-api-tls\") pod \"maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk\" (UID: \"3b920554-398b-4dce-b517-4374288dc369\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" Jul 22 02:15:19.545019 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:15:19.544941 2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-serving-cert: secret "maas-api-e2e-api-a-ae6764-serving-cert" not found Jul 22 02:15:19.545019 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:15:19.545007 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b920554-398b-4dce-b517-4374288dc369-maas-api-tls podName:3b920554-398b-4dce-b517-4374288dc369 nodeName:}" failed. No retries permitted until 2026-07-22 02:15:20.544991443 +0000 UTC m=+2305.374239770 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3b920554-398b-4dce-b517-4374288dc369-maas-api-tls") pod "maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" (UID: "3b920554-398b-4dce-b517-4374288dc369") : secret "maas-api-e2e-api-a-ae6764-serving-cert" not found Jul 22 02:15:20.555996 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:20.555954 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b920554-398b-4dce-b517-4374288dc369-maas-api-tls\") pod \"maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk\" (UID: \"3b920554-398b-4dce-b517-4374288dc369\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" Jul 22 02:15:20.558256 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:20.558234 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b920554-398b-4dce-b517-4374288dc369-maas-api-tls\") pod \"maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk\" (UID: \"3b920554-398b-4dce-b517-4374288dc369\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" Jul 22 02:15:20.708748 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:20.708709 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" Jul 22 02:15:20.859340 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:20.859309 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk"] Jul 22 02:15:20.862328 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:15:20.862298 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b920554_398b_4dce_b517_4374288dc369.slice/crio-340df422899e804dbb67d66e9a84dfe0c758bf7d03066a38149f5d1f74346833 WatchSource:0}: Error finding container 340df422899e804dbb67d66e9a84dfe0c758bf7d03066a38149f5d1f74346833: Status 404 returned error can't find the container with id 340df422899e804dbb67d66e9a84dfe0c758bf7d03066a38149f5d1f74346833 Jul 22 02:15:21.697801 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:21.697707 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" event={"ID":"3b920554-398b-4dce-b517-4374288dc369","Type":"ContainerStarted","Data":"a90f362f6b1ec99d6eee4e73b1127d6dc069e433b6751b617b9ce8c5ee5d58a2"} Jul 22 02:15:21.697801 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:21.697762 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" event={"ID":"3b920554-398b-4dce-b517-4374288dc369","Type":"ContainerStarted","Data":"340df422899e804dbb67d66e9a84dfe0c758bf7d03066a38149f5d1f74346833"} Jul 22 02:15:21.697801 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:21.697794 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" Jul 22 02:15:21.717600 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:21.717548 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" podStartSLOduration=3.1646783 podStartE2EDuration="3.717514624s" totalImagesPullingTime="552.836324ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:15:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:15:21.715938258 +0000 UTC m=+2306.545186605" watchObservedRunningTime="2026-07-22 02:15:21.717514624 +0000 UTC m=+2306.546763038" Jul 22 02:15:27.236374 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:27.232523 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz"] Jul 22 02:15:27.247389 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:27.244935 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" Jul 22 02:15:27.260859 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:27.256788 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-703b60-serving-cert\"" Jul 22 02:15:27.266640 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:27.266588 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz"] Jul 22 02:15:27.327848 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:27.327802 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5l6n\" (UniqueName: \"kubernetes.io/projected/0487364e-5160-48b1-b6fd-ee58f147c9f5-kube-api-access-s5l6n\") pod \"maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz\" (UID: \"0487364e-5160-48b1-b6fd-ee58f147c9f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" Jul 22 02:15:27.328035 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:27.327880 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0487364e-5160-48b1-b6fd-ee58f147c9f5-maas-api-tls\") pod \"maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz\" (UID: \"0487364e-5160-48b1-b6fd-ee58f147c9f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" Jul 22 02:15:27.328035 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:27.327927 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0487364e-5160-48b1-b6fd-ee58f147c9f5-openshift-service-ca\") pod \"maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz\" (UID: \"0487364e-5160-48b1-b6fd-ee58f147c9f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" Jul 22 02:15:27.429580 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:27.429541 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0487364e-5160-48b1-b6fd-ee58f147c9f5-openshift-service-ca\") pod \"maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz\" (UID: \"0487364e-5160-48b1-b6fd-ee58f147c9f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" Jul 22 02:15:27.429773 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:27.429659 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5l6n\" (UniqueName: \"kubernetes.io/projected/0487364e-5160-48b1-b6fd-ee58f147c9f5-kube-api-access-s5l6n\") pod \"maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz\" (UID: \"0487364e-5160-48b1-b6fd-ee58f147c9f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" Jul 22 02:15:27.429773 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:27.429738 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0487364e-5160-48b1-b6fd-ee58f147c9f5-maas-api-tls\") pod \"maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz\" (UID: \"0487364e-5160-48b1-b6fd-ee58f147c9f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" Jul 22 02:15:27.429875 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:15:27.429857 2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-serving-cert: secret "maas-api-e2e-api-b-703b60-serving-cert" not found Jul 22 02:15:27.429947 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:15:27.429936 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0487364e-5160-48b1-b6fd-ee58f147c9f5-maas-api-tls podName:0487364e-5160-48b1-b6fd-ee58f147c9f5 nodeName:}" failed. No retries permitted until 2026-07-22 02:15:27.929911976 +0000 UTC m=+2312.759160321 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0487364e-5160-48b1-b6fd-ee58f147c9f5-maas-api-tls") pod "maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" (UID: "0487364e-5160-48b1-b6fd-ee58f147c9f5") : secret "maas-api-e2e-api-b-703b60-serving-cert" not found Jul 22 02:15:27.430446 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:27.430416 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0487364e-5160-48b1-b6fd-ee58f147c9f5-openshift-service-ca\") pod \"maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz\" (UID: \"0487364e-5160-48b1-b6fd-ee58f147c9f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" Jul 22 02:15:27.456767 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:27.456731 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5l6n\" (UniqueName: \"kubernetes.io/projected/0487364e-5160-48b1-b6fd-ee58f147c9f5-kube-api-access-s5l6n\") pod \"maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz\" (UID: \"0487364e-5160-48b1-b6fd-ee58f147c9f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" Jul 22 02:15:27.709490 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:27.709454 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" Jul 22 02:15:27.935449 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:27.935389 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0487364e-5160-48b1-b6fd-ee58f147c9f5-maas-api-tls\") pod \"maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz\" (UID: \"0487364e-5160-48b1-b6fd-ee58f147c9f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" Jul 22 02:15:27.948998 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:27.948950 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0487364e-5160-48b1-b6fd-ee58f147c9f5-maas-api-tls\") pod \"maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz\" (UID: \"0487364e-5160-48b1-b6fd-ee58f147c9f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" Jul 22 02:15:28.173311 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:28.173275 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" Jul 22 02:15:28.356741 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:28.351076 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz"] Jul 22 02:15:28.363583 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:15:28.363553 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0487364e_5160_48b1_b6fd_ee58f147c9f5.slice/crio-9165af7a8fc2359da02abc69f60b8ae6c34537e25f6fe1b0bfeabbfe6612dcd8 WatchSource:0}: Error finding container 9165af7a8fc2359da02abc69f60b8ae6c34537e25f6fe1b0bfeabbfe6612dcd8: Status 404 returned error can't find the container with id 9165af7a8fc2359da02abc69f60b8ae6c34537e25f6fe1b0bfeabbfe6612dcd8 Jul 22 02:15:28.741256 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:28.741215 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" event={"ID":"0487364e-5160-48b1-b6fd-ee58f147c9f5","Type":"ContainerStarted","Data":"7990b46cb969d5b32a46ee32c9d1694a13d26cebf67acaef0c38d52b9c6a4506"} Jul 22 02:15:28.741450 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:28.741284 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" event={"ID":"0487364e-5160-48b1-b6fd-ee58f147c9f5","Type":"ContainerStarted","Data":"9165af7a8fc2359da02abc69f60b8ae6c34537e25f6fe1b0bfeabbfe6612dcd8"} Jul 22 02:15:28.742624 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:28.742210 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" Jul 22 02:15:28.777646 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:28.777590 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" podStartSLOduration=1.4685757640000001 podStartE2EDuration="1.77756762s" totalImagesPullingTime="308.991856ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:15:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:15:28.775427937 +0000 UTC m=+2313.604676285" watchObservedRunningTime="2026-07-22 02:15:28.77756762 +0000 UTC m=+2313.606815969" Jul 22 02:15:31.760056 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:31.760018 2559 generic.go:356] "Generic (PLEG): container finished" podID="2f341a5b-981d-4adf-aab2-353d5c890d7d" containerID="4a4760eaef0f7994b85941b04141d3e9013a20ae9aa7f99260cfd5f1193d0b60" exitCode=0 Jul 22 02:15:31.760525 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:31.760091 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" event={"ID":"2f341a5b-981d-4adf-aab2-353d5c890d7d","Type":"ContainerDied","Data":"4a4760eaef0f7994b85941b04141d3e9013a20ae9aa7f99260cfd5f1193d0b60"} Jul 22 02:15:31.760525 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:31.760520 2559 scope.go:117] "RemoveContainer" containerID="4a4760eaef0f7994b85941b04141d3e9013a20ae9aa7f99260cfd5f1193d0b60" Jul 22 02:15:31.761586 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:31.761556 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:15:31.761704 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:31.761641 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:15:31.761704 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:31.761690 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:15:32.677994 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:32.677956 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:32.767244 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:32.767211 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" event={"ID":"2f341a5b-981d-4adf-aab2-353d5c890d7d","Type":"ContainerStarted","Data":"0d4b04832b6f4b390e5ae03423b151c84a1b17603cbc55b2fd89fe663b81a7a2"} Jul 22 02:15:33.700046 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:33.700004 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:33.701414 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:33.701377 2559 patch_prober.go:28] interesting pod/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 22 02:15:33.701577 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:33.701443 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" podUID="2f341a5b-981d-4adf-aab2-353d5c890d7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 22 02:15:34.700371 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:34.700295 2559 patch_prober.go:28] interesting pod/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 22 02:15:34.700770 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:34.700400 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" podUID="2f341a5b-981d-4adf-aab2-353d5c890d7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 22 02:15:35.700039 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:35.700002 2559 patch_prober.go:28] interesting pod/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 22 02:15:35.700226 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:35.700061 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" podUID="2f341a5b-981d-4adf-aab2-353d5c890d7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 22 02:15:35.761128 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:35.761096 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" Jul 22 02:15:36.703097 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:36.703068 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:36.703395 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:36.703375 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:36.704015 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:36.703994 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:15:55.803663 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:55.803455 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk"] Jul 22 02:15:55.804927 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:55.804469 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" podUID="3b920554-398b-4dce-b517-4374288dc369" containerName="maas-api" containerID="cri-o://a90f362f6b1ec99d6eee4e73b1127d6dc069e433b6751b617b9ce8c5ee5d58a2" gracePeriod=30 Jul 22 02:15:56.063484 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:56.063420 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" Jul 22 02:15:56.153996 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:56.153961 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-898dt\" (UniqueName: \"kubernetes.io/projected/3b920554-398b-4dce-b517-4374288dc369-kube-api-access-898dt\") pod \"3b920554-398b-4dce-b517-4374288dc369\" (UID: \"3b920554-398b-4dce-b517-4374288dc369\") " Jul 22 02:15:56.154179 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:56.154029 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b920554-398b-4dce-b517-4374288dc369-maas-api-tls\") pod \"3b920554-398b-4dce-b517-4374288dc369\" (UID: \"3b920554-398b-4dce-b517-4374288dc369\") " Jul 22 02:15:56.154179 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:56.154092 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b920554-398b-4dce-b517-4374288dc369-openshift-service-ca\") pod \"3b920554-398b-4dce-b517-4374288dc369\" (UID: \"3b920554-398b-4dce-b517-4374288dc369\") " Jul 22 02:15:56.154486 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:56.154453 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b920554-398b-4dce-b517-4374288dc369-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3b920554-398b-4dce-b517-4374288dc369" (UID: "3b920554-398b-4dce-b517-4374288dc369"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:15:56.156205 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:56.156180 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b920554-398b-4dce-b517-4374288dc369-kube-api-access-898dt" (OuterVolumeSpecName: "kube-api-access-898dt") pod "3b920554-398b-4dce-b517-4374288dc369" (UID: "3b920554-398b-4dce-b517-4374288dc369"). InnerVolumeSpecName "kube-api-access-898dt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:15:56.156297 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:56.156183 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b920554-398b-4dce-b517-4374288dc369-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3b920554-398b-4dce-b517-4374288dc369" (UID: "3b920554-398b-4dce-b517-4374288dc369"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:15:56.255154 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:56.255112 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-898dt\" (UniqueName: \"kubernetes.io/projected/3b920554-398b-4dce-b517-4374288dc369-kube-api-access-898dt\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:15:56.255154 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:56.255149 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b920554-398b-4dce-b517-4374288dc369-maas-api-tls\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:15:56.255154 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:56.255161 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b920554-398b-4dce-b517-4374288dc369-openshift-service-ca\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:15:56.891253 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:56.891221 2559 generic.go:356] "Generic (PLEG): container finished" podID="3b920554-398b-4dce-b517-4374288dc369" containerID="a90f362f6b1ec99d6eee4e73b1127d6dc069e433b6751b617b9ce8c5ee5d58a2" exitCode=0 Jul 22 02:15:56.891873 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:56.891266 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" Jul 22 02:15:56.891873 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:56.891271 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" event={"ID":"3b920554-398b-4dce-b517-4374288dc369","Type":"ContainerDied","Data":"a90f362f6b1ec99d6eee4e73b1127d6dc069e433b6751b617b9ce8c5ee5d58a2"} Jul 22 02:15:56.891873 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:56.891330 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk" event={"ID":"3b920554-398b-4dce-b517-4374288dc369","Type":"ContainerDied","Data":"340df422899e804dbb67d66e9a84dfe0c758bf7d03066a38149f5d1f74346833"} Jul 22 02:15:56.891873 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:56.891402 2559 scope.go:117] "RemoveContainer" containerID="a90f362f6b1ec99d6eee4e73b1127d6dc069e433b6751b617b9ce8c5ee5d58a2" Jul 22 02:15:56.900942 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:56.900922 2559 scope.go:117] "RemoveContainer" containerID="a90f362f6b1ec99d6eee4e73b1127d6dc069e433b6751b617b9ce8c5ee5d58a2" Jul 22 02:15:56.901231 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:15:56.901212 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a90f362f6b1ec99d6eee4e73b1127d6dc069e433b6751b617b9ce8c5ee5d58a2\": container with ID starting with a90f362f6b1ec99d6eee4e73b1127d6dc069e433b6751b617b9ce8c5ee5d58a2 not found: ID does not exist" containerID="a90f362f6b1ec99d6eee4e73b1127d6dc069e433b6751b617b9ce8c5ee5d58a2" Jul 22 02:15:56.901300 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:56.901240 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a90f362f6b1ec99d6eee4e73b1127d6dc069e433b6751b617b9ce8c5ee5d58a2"} err="failed to get container status \"a90f362f6b1ec99d6eee4e73b1127d6dc069e433b6751b617b9ce8c5ee5d58a2\": rpc error: code = NotFound desc = could not find container \"a90f362f6b1ec99d6eee4e73b1127d6dc069e433b6751b617b9ce8c5ee5d58a2\": container with ID starting with a90f362f6b1ec99d6eee4e73b1127d6dc069e433b6751b617b9ce8c5ee5d58a2 not found: ID does not exist" Jul 22 02:15:56.915276 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:56.915242 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk"] Jul 22 02:15:56.917325 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:56.917296 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ae6764-59bfc98bcd-crzxk"] Jul 22 02:15:57.625790 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:15:57.625752 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b920554-398b-4dce-b517-4374288dc369" path="/var/lib/kubelet/pods/3b920554-398b-4dce-b517-4374288dc369/volumes" Jul 22 02:16:06.862733 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:06.862688 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x"] Jul 22 02:16:06.863220 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:06.862912 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" podUID="2f341a5b-981d-4adf-aab2-353d5c890d7d" containerName="istio-proxy" containerID="cri-o://0d4b04832b6f4b390e5ae03423b151c84a1b17603cbc55b2fd89fe663b81a7a2" gracePeriod=30 Jul 22 02:16:11.978280 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:11.978233 2559 generic.go:356] "Generic (PLEG): container finished" podID="2f341a5b-981d-4adf-aab2-353d5c890d7d" containerID="0d4b04832b6f4b390e5ae03423b151c84a1b17603cbc55b2fd89fe663b81a7a2" exitCode=0 Jul 22 02:16:11.978642 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:11.978296 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" event={"ID":"2f341a5b-981d-4adf-aab2-353d5c890d7d","Type":"ContainerDied","Data":"0d4b04832b6f4b390e5ae03423b151c84a1b17603cbc55b2fd89fe663b81a7a2"} Jul 22 02:16:11.978642 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:11.978418 2559 scope.go:117] "RemoveContainer" containerID="4a4760eaef0f7994b85941b04141d3e9013a20ae9aa7f99260cfd5f1193d0b60" Jul 22 02:16:12.114033 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.114003 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:16:12.115877 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.115855 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-data\") pod \"2f341a5b-981d-4adf-aab2-353d5c890d7d\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " Jul 22 02:16:12.115998 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.115884 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-credential-socket\") pod \"2f341a5b-981d-4adf-aab2-353d5c890d7d\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " Jul 22 02:16:12.115998 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.115904 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f341a5b-981d-4adf-aab2-353d5c890d7d-istiod-ca-cert\") pod \"2f341a5b-981d-4adf-aab2-353d5c890d7d\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " Jul 22 02:16:12.115998 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.115933 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-workload-certs\") pod \"2f341a5b-981d-4adf-aab2-353d5c890d7d\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " Jul 22 02:16:12.115998 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.115956 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-token\") pod \"2f341a5b-981d-4adf-aab2-353d5c890d7d\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " Jul 22 02:16:12.115998 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.115980 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-envoy\") pod \"2f341a5b-981d-4adf-aab2-353d5c890d7d\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " Jul 22 02:16:12.116259 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.116013 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-workload-socket\") pod \"2f341a5b-981d-4adf-aab2-353d5c890d7d\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " Jul 22 02:16:12.116259 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.116088 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mrrb\" (UniqueName: \"kubernetes.io/projected/2f341a5b-981d-4adf-aab2-353d5c890d7d-kube-api-access-4mrrb\") pod \"2f341a5b-981d-4adf-aab2-353d5c890d7d\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " Jul 22 02:16:12.116259 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.116129 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-podinfo\") pod \"2f341a5b-981d-4adf-aab2-353d5c890d7d\" (UID: \"2f341a5b-981d-4adf-aab2-353d5c890d7d\") " Jul 22 02:16:12.116259 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.116180 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2f341a5b-981d-4adf-aab2-353d5c890d7d" (UID: "2f341a5b-981d-4adf-aab2-353d5c890d7d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:16:12.116259 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.116208 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2f341a5b-981d-4adf-aab2-353d5c890d7d" (UID: "2f341a5b-981d-4adf-aab2-353d5c890d7d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:16:12.116525 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.116326 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f341a5b-981d-4adf-aab2-353d5c890d7d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2f341a5b-981d-4adf-aab2-353d5c890d7d" (UID: "2f341a5b-981d-4adf-aab2-353d5c890d7d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:16:12.116525 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.116373 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2f341a5b-981d-4adf-aab2-353d5c890d7d" (UID: "2f341a5b-981d-4adf-aab2-353d5c890d7d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:16:12.116633 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.116531 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-credential-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:16:12.116633 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.116552 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f341a5b-981d-4adf-aab2-353d5c890d7d-istiod-ca-cert\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:16:12.116633 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.116568 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-workload-certs\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:16:12.116633 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.116581 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-workload-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:16:12.117045 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.116961 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-data" (OuterVolumeSpecName: "istio-data") pod "2f341a5b-981d-4adf-aab2-353d5c890d7d" (UID: "2f341a5b-981d-4adf-aab2-353d5c890d7d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:16:12.118277 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.118251 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2f341a5b-981d-4adf-aab2-353d5c890d7d" (UID: "2f341a5b-981d-4adf-aab2-353d5c890d7d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:16:12.118424 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.118380 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2f341a5b-981d-4adf-aab2-353d5c890d7d" (UID: "2f341a5b-981d-4adf-aab2-353d5c890d7d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:16:12.118708 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.118689 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f341a5b-981d-4adf-aab2-353d5c890d7d-kube-api-access-4mrrb" (OuterVolumeSpecName: "kube-api-access-4mrrb") pod "2f341a5b-981d-4adf-aab2-353d5c890d7d" (UID: "2f341a5b-981d-4adf-aab2-353d5c890d7d"). InnerVolumeSpecName "kube-api-access-4mrrb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:16:12.118708 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.118695 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-token" (OuterVolumeSpecName: "istio-token") pod "2f341a5b-981d-4adf-aab2-353d5c890d7d" (UID: "2f341a5b-981d-4adf-aab2-353d5c890d7d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:16:12.217245 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.217201 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mrrb\" (UniqueName: \"kubernetes.io/projected/2f341a5b-981d-4adf-aab2-353d5c890d7d-kube-api-access-4mrrb\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:16:12.217245 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.217238 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-podinfo\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:16:12.217245 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.217249 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-data\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:16:12.217245 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.217256 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-token\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:16:12.217560 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.217264 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f341a5b-981d-4adf-aab2-353d5c890d7d-istio-envoy\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:16:12.985254 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.985217 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" event={"ID":"2f341a5b-981d-4adf-aab2-353d5c890d7d","Type":"ContainerDied","Data":"8a336dedc0f1259c86958a913c9f658b252563b2aa1836506898aa150ef7096b"} Jul 22 02:16:12.985254 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.985249 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x" Jul 22 02:16:12.985778 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:12.985288 2559 scope.go:117] "RemoveContainer" containerID="0d4b04832b6f4b390e5ae03423b151c84a1b17603cbc55b2fd89fe663b81a7a2" Jul 22 02:16:13.010085 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:13.010051 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x"] Jul 22 02:16:13.014526 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:13.014490 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ae6764-openshift-default-5cc6ccfbd9-rh54x"] Jul 22 02:16:13.626798 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:13.626768 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f341a5b-981d-4adf-aab2-353d5c890d7d" path="/var/lib/kubelet/pods/2f341a5b-981d-4adf-aab2-353d5c890d7d/volumes" Jul 22 02:16:22.835184 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:22.835147 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz"] Jul 22 02:16:22.836068 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:22.835997 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" podUID="0487364e-5160-48b1-b6fd-ee58f147c9f5" containerName="maas-api" containerID="cri-o://7990b46cb969d5b32a46ee32c9d1694a13d26cebf67acaef0c38d52b9c6a4506" gracePeriod=30 Jul 22 02:16:23.038816 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:23.038778 2559 generic.go:356] "Generic (PLEG): container finished" podID="0487364e-5160-48b1-b6fd-ee58f147c9f5" containerID="7990b46cb969d5b32a46ee32c9d1694a13d26cebf67acaef0c38d52b9c6a4506" exitCode=0 Jul 22 02:16:23.038978 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:23.038853 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" event={"ID":"0487364e-5160-48b1-b6fd-ee58f147c9f5","Type":"ContainerDied","Data":"7990b46cb969d5b32a46ee32c9d1694a13d26cebf67acaef0c38d52b9c6a4506"} Jul 22 02:16:23.090604 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:23.090536 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" Jul 22 02:16:23.227485 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:23.227456 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0487364e-5160-48b1-b6fd-ee58f147c9f5-maas-api-tls\") pod \"0487364e-5160-48b1-b6fd-ee58f147c9f5\" (UID: \"0487364e-5160-48b1-b6fd-ee58f147c9f5\") " Jul 22 02:16:23.227654 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:23.227641 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0487364e-5160-48b1-b6fd-ee58f147c9f5-openshift-service-ca\") pod \"0487364e-5160-48b1-b6fd-ee58f147c9f5\" (UID: \"0487364e-5160-48b1-b6fd-ee58f147c9f5\") " Jul 22 02:16:23.227694 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:23.227680 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5l6n\" (UniqueName: \"kubernetes.io/projected/0487364e-5160-48b1-b6fd-ee58f147c9f5-kube-api-access-s5l6n\") pod \"0487364e-5160-48b1-b6fd-ee58f147c9f5\" (UID: \"0487364e-5160-48b1-b6fd-ee58f147c9f5\") " Jul 22 02:16:23.228052 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:23.228022 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0487364e-5160-48b1-b6fd-ee58f147c9f5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0487364e-5160-48b1-b6fd-ee58f147c9f5" (UID: "0487364e-5160-48b1-b6fd-ee58f147c9f5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:16:23.229549 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:23.229529 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0487364e-5160-48b1-b6fd-ee58f147c9f5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0487364e-5160-48b1-b6fd-ee58f147c9f5" (UID: "0487364e-5160-48b1-b6fd-ee58f147c9f5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:16:23.229772 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:23.229750 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0487364e-5160-48b1-b6fd-ee58f147c9f5-kube-api-access-s5l6n" (OuterVolumeSpecName: "kube-api-access-s5l6n") pod "0487364e-5160-48b1-b6fd-ee58f147c9f5" (UID: "0487364e-5160-48b1-b6fd-ee58f147c9f5"). InnerVolumeSpecName "kube-api-access-s5l6n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:16:23.328964 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:23.328929 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0487364e-5160-48b1-b6fd-ee58f147c9f5-openshift-service-ca\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:16:23.328964 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:23.328957 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5l6n\" (UniqueName: \"kubernetes.io/projected/0487364e-5160-48b1-b6fd-ee58f147c9f5-kube-api-access-s5l6n\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:16:23.328964 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:23.328968 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0487364e-5160-48b1-b6fd-ee58f147c9f5-maas-api-tls\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:16:24.044960 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:24.044875 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" Jul 22 02:16:24.045450 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:24.044878 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz" event={"ID":"0487364e-5160-48b1-b6fd-ee58f147c9f5","Type":"ContainerDied","Data":"9165af7a8fc2359da02abc69f60b8ae6c34537e25f6fe1b0bfeabbfe6612dcd8"} Jul 22 02:16:24.045450 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:24.045015 2559 scope.go:117] "RemoveContainer" containerID="7990b46cb969d5b32a46ee32c9d1694a13d26cebf67acaef0c38d52b9c6a4506" Jul 22 02:16:24.063376 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:24.063332 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz"] Jul 22 02:16:24.067732 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:24.067707 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-703b60-5fc454fb7d-vjmkz"] Jul 22 02:16:25.625858 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:25.625820 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0487364e-5160-48b1-b6fd-ee58f147c9f5" path="/var/lib/kubelet/pods/0487364e-5160-48b1-b6fd-ee58f147c9f5/volumes" Jul 22 02:16:35.276212 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.276178 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v"] Jul 22 02:16:35.276742 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.276629 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b920554-398b-4dce-b517-4374288dc369" containerName="maas-api" Jul 22 02:16:35.276742 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.276645 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b920554-398b-4dce-b517-4374288dc369" containerName="maas-api" Jul 22 02:16:35.276742 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.276666 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f341a5b-981d-4adf-aab2-353d5c890d7d" containerName="istio-proxy" Jul 22 02:16:35.276742 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.276671 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f341a5b-981d-4adf-aab2-353d5c890d7d" containerName="istio-proxy" Jul 22 02:16:35.276742 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.276679 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f341a5b-981d-4adf-aab2-353d5c890d7d" containerName="istio-proxy" Jul 22 02:16:35.276742 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.276687 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f341a5b-981d-4adf-aab2-353d5c890d7d" containerName="istio-proxy" Jul 22 02:16:35.276742 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.276698 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0487364e-5160-48b1-b6fd-ee58f147c9f5" containerName="maas-api" Jul 22 02:16:35.276742 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.276704 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0487364e-5160-48b1-b6fd-ee58f147c9f5" containerName="maas-api" Jul 22 02:16:35.276994 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.276760 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f341a5b-981d-4adf-aab2-353d5c890d7d" containerName="istio-proxy" Jul 22 02:16:35.276994 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.276774 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0487364e-5160-48b1-b6fd-ee58f147c9f5" containerName="maas-api" Jul 22 02:16:35.276994 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.276784 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b920554-398b-4dce-b517-4374288dc369" containerName="maas-api" Jul 22 02:16:35.279802 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.279785 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.282315 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.282291 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-eba9c8-openshift-default-dockercfg-m824d\"" Jul 22 02:16:35.291831 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.291809 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v"] Jul 22 02:16:35.449310 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.449270 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-workload-socket\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.449310 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.449320 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-envoy\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.449609 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.449375 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-data\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.449609 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.449401 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-podinfo\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.449609 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.449459 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15d629b2-d3a1-4423-8df3-39c6661b4a71-istiod-ca-cert\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.449609 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.449539 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-token\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.449609 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.449579 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbtsx\" (UniqueName: \"kubernetes.io/projected/15d629b2-d3a1-4423-8df3-39c6661b4a71-kube-api-access-jbtsx\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.449843 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.449644 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-credential-socket\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.449843 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.449695 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-workload-certs\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.550331 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.550244 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-credential-socket\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.550331 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.550287 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-workload-certs\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.550564 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.550450 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-workload-socket\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.550564 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.550507 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-envoy\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.550564 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.550539 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-data\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.550742 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.550572 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-podinfo\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.550742 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.550650 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15d629b2-d3a1-4423-8df3-39c6661b4a71-istiod-ca-cert\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.550742 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.550665 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-workload-certs\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.550742 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.550695 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-token\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.550742 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.550719 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbtsx\" (UniqueName: \"kubernetes.io/projected/15d629b2-d3a1-4423-8df3-39c6661b4a71-kube-api-access-jbtsx\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.551005 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.550719 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-credential-socket\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.551005 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.550802 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-workload-socket\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.551005 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.550865 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-data\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.551411 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.551387 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15d629b2-d3a1-4423-8df3-39c6661b4a71-istiod-ca-cert\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.552702 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.552678 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-envoy\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.553041 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.553023 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-podinfo\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.559619 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.559597 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-token\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.559832 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.559809 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbtsx\" (UniqueName: \"kubernetes.io/projected/15d629b2-d3a1-4423-8df3-39c6661b4a71-kube-api-access-jbtsx\") pod \"e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.590459 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.590423 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:35.732640 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.732613 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v"] Jul 22 02:16:35.735415 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:16:35.735379 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15d629b2_d3a1_4423_8df3_39c6661b4a71.slice/crio-b1315fd9050fa5d1e7bd533fd31307e270f4bd6c416bd03f47cf8060e6e2fecc WatchSource:0}: Error finding container b1315fd9050fa5d1e7bd533fd31307e270f4bd6c416bd03f47cf8060e6e2fecc: Status 404 returned error can't find the container with id b1315fd9050fa5d1e7bd533fd31307e270f4bd6c416bd03f47cf8060e6e2fecc Jul 22 02:16:35.738336 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.738290 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:16:35.738454 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.738385 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:16:35.738454 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:35.738421 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:16:36.106632 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:36.106596 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" event={"ID":"15d629b2-d3a1-4423-8df3-39c6661b4a71","Type":"ContainerStarted","Data":"555a004006f42578b80c924da4e43688357af90150405f6b26af40bef62edfa3"} Jul 22 02:16:36.106857 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:36.106646 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" event={"ID":"15d629b2-d3a1-4423-8df3-39c6661b4a71","Type":"ContainerStarted","Data":"b1315fd9050fa5d1e7bd533fd31307e270f4bd6c416bd03f47cf8060e6e2fecc"} Jul 22 02:16:36.137373 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:36.137304 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" podStartSLOduration=1.137288586 podStartE2EDuration="1.137288586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:16:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:16:36.137138981 +0000 UTC m=+2380.966387330" watchObservedRunningTime="2026-07-22 02:16:36.137288586 +0000 UTC m=+2380.966536934" Jul 22 02:16:36.590575 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:36.590540 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:36.595653 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:36.595625 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:37.111922 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:37.111886 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:37.112772 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:37.112753 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:38.402008 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:38.401968 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt"] Jul 22 02:16:38.402595 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:38.402575 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f341a5b-981d-4adf-aab2-353d5c890d7d" containerName="istio-proxy" Jul 22 02:16:38.405882 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:38.405858 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" Jul 22 02:16:38.409544 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:38.409521 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-eba9c8-serving-cert\"" Jul 22 02:16:38.409800 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:38.409781 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-49lpj\"" Jul 22 02:16:38.409916 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:38.409783 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 02:16:38.410561 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:38.410541 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 22 02:16:38.416988 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:38.416961 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt"] Jul 22 02:16:38.584387 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:38.584355 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj46s\" (UniqueName: \"kubernetes.io/projected/f69a1a75-7dc8-435a-ad4d-1559b859cff9-kube-api-access-qj46s\") pod \"maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt\" (UID: \"f69a1a75-7dc8-435a-ad4d-1559b859cff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" Jul 22 02:16:38.584566 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:38.584398 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f69a1a75-7dc8-435a-ad4d-1559b859cff9-maas-api-tls\") pod \"maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt\" (UID: \"f69a1a75-7dc8-435a-ad4d-1559b859cff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" Jul 22 02:16:38.584566 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:38.584419 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f69a1a75-7dc8-435a-ad4d-1559b859cff9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt\" (UID: \"f69a1a75-7dc8-435a-ad4d-1559b859cff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" Jul 22 02:16:38.685466 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:38.685378 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj46s\" (UniqueName: \"kubernetes.io/projected/f69a1a75-7dc8-435a-ad4d-1559b859cff9-kube-api-access-qj46s\") pod \"maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt\" (UID: \"f69a1a75-7dc8-435a-ad4d-1559b859cff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" Jul 22 02:16:38.685466 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:38.685423 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f69a1a75-7dc8-435a-ad4d-1559b859cff9-maas-api-tls\") pod \"maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt\" (UID: \"f69a1a75-7dc8-435a-ad4d-1559b859cff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" Jul 22 02:16:38.685667 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:38.685524 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f69a1a75-7dc8-435a-ad4d-1559b859cff9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt\" (UID: \"f69a1a75-7dc8-435a-ad4d-1559b859cff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" Jul 22 02:16:38.686221 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:38.686194 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f69a1a75-7dc8-435a-ad4d-1559b859cff9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt\" (UID: \"f69a1a75-7dc8-435a-ad4d-1559b859cff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" Jul 22 02:16:38.687903 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:38.687881 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f69a1a75-7dc8-435a-ad4d-1559b859cff9-maas-api-tls\") pod \"maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt\" (UID: \"f69a1a75-7dc8-435a-ad4d-1559b859cff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" Jul 22 02:16:38.694602 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:38.694572 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj46s\" (UniqueName: \"kubernetes.io/projected/f69a1a75-7dc8-435a-ad4d-1559b859cff9-kube-api-access-qj46s\") pod \"maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt\" (UID: \"f69a1a75-7dc8-435a-ad4d-1559b859cff9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" Jul 22 02:16:38.718552 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:38.718522 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" Jul 22 02:16:38.858680 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:38.858650 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt"] Jul 22 02:16:38.864801 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:16:38.862527 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf69a1a75_7dc8_435a_ad4d_1559b859cff9.slice/crio-35bb30097af5f1c0691fd96488d0efb4755a593d4dc6478cd362b512ffde51c5 WatchSource:0}: Error finding container 35bb30097af5f1c0691fd96488d0efb4755a593d4dc6478cd362b512ffde51c5: Status 404 returned error can't find the container with id 35bb30097af5f1c0691fd96488d0efb4755a593d4dc6478cd362b512ffde51c5 Jul 22 02:16:39.123126 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:39.123092 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" event={"ID":"f69a1a75-7dc8-435a-ad4d-1559b859cff9","Type":"ContainerStarted","Data":"35bb30097af5f1c0691fd96488d0efb4755a593d4dc6478cd362b512ffde51c5"} Jul 22 02:16:40.128956 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:40.128923 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" event={"ID":"f69a1a75-7dc8-435a-ad4d-1559b859cff9","Type":"ContainerStarted","Data":"7afa3cfd24042b49d1780385df9a69a2633b648a4b13ca341f3284ceeeaf7809"} Jul 22 02:16:40.129491 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:40.128987 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" Jul 22 02:16:40.151752 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:40.151701 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" podStartSLOduration=1.7427347819999999 podStartE2EDuration="2.151680586s" totalImagesPullingTime="408.945804ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:16:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:16:40.150532424 +0000 UTC m=+2384.979780772" watchObservedRunningTime="2026-07-22 02:16:40.151680586 +0000 UTC m=+2384.980928938" Jul 22 02:16:43.671989 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.671954 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w"] Jul 22 02:16:43.675723 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.675702 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.678253 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.678210 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-c6960c-openshift-default-dockercfg-hxjlc\"" Jul 22 02:16:43.689939 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.689913 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w"] Jul 22 02:16:43.734372 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.734310 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw5pt\" (UniqueName: \"kubernetes.io/projected/7ee7c7da-a2e5-4f23-a427-f2246f721b02-kube-api-access-jw5pt\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.734372 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.734366 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-podinfo\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.734572 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.734393 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-token\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.734572 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.734427 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istiod-ca-cert\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.734572 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.734519 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-data\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.734719 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.734611 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-workload-socket\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.734719 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.734635 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-credential-socket\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.734719 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.734656 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-workload-certs\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.734719 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.734684 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-envoy\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.835724 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.835684 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw5pt\" (UniqueName: \"kubernetes.io/projected/7ee7c7da-a2e5-4f23-a427-f2246f721b02-kube-api-access-jw5pt\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.835724 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.835726 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-podinfo\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.835954 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.835752 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-token\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.835954 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.835772 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istiod-ca-cert\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.835954 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.835812 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-data\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.835954 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.835910 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-workload-socket\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.835954 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.835937 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-credential-socket\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.836194 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.835961 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-workload-certs\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.836194 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.835988 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-envoy\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.836432 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.836403 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-credential-socket\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.836606 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.836581 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istiod-ca-cert\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.836732 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.836457 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-workload-socket\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.836732 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.836530 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-workload-certs\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.836732 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.836713 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-data\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.838225 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.838204 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-envoy\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.838354 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.838240 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-podinfo\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.847394 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.847374 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-token\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.847581 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.847563 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw5pt\" (UniqueName: \"kubernetes.io/projected/7ee7c7da-a2e5-4f23-a427-f2246f721b02-kube-api-access-jw5pt\") pod \"e2e-shared-b-c6960c-openshift-default-895878c-zmr6w\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:43.986706 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:43.986617 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:44.129338 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:44.129309 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w"] Jul 22 02:16:44.131335 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:16:44.131310 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ee7c7da_a2e5_4f23_a427_f2246f721b02.slice/crio-498ff10f4eebb29b7360b6bdebfb87e8247449a2c2481a83596faa4ad1442b05 WatchSource:0}: Error finding container 498ff10f4eebb29b7360b6bdebfb87e8247449a2c2481a83596faa4ad1442b05: Status 404 returned error can't find the container with id 498ff10f4eebb29b7360b6bdebfb87e8247449a2c2481a83596faa4ad1442b05 Jul 22 02:16:44.133586 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:44.133553 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:16:44.133646 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:44.133618 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:16:44.133688 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:44.133646 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:16:44.151239 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:44.151215 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" event={"ID":"7ee7c7da-a2e5-4f23-a427-f2246f721b02","Type":"ContainerStarted","Data":"498ff10f4eebb29b7360b6bdebfb87e8247449a2c2481a83596faa4ad1442b05"} Jul 22 02:16:45.158027 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:45.157988 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" event={"ID":"7ee7c7da-a2e5-4f23-a427-f2246f721b02","Type":"ContainerStarted","Data":"e005dfd9ec244c6493a6d8232afece96c81cf268d6b9a58c1bd3ffc8027de8ec"} Jul 22 02:16:45.191853 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:45.191799 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" podStartSLOduration=2.1917833780000002 podStartE2EDuration="2.191783378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:16:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:16:45.190248992 +0000 UTC m=+2390.019497340" watchObservedRunningTime="2026-07-22 02:16:45.191783378 +0000 UTC m=+2390.021031798" Jul 22 02:16:45.987699 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:45.987656 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:45.993332 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:45.993301 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:46.140817 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:46.140787 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" Jul 22 02:16:46.163124 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:46.163088 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:46.164234 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:46.164205 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:16:53.153573 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:53.153526 2559 patch_prober.go:28] interesting pod/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 02:16:53.153977 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:53.153614 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" podUID="15d629b2-d3a1-4423-8df3-39c6661b4a71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 02:16:54.206481 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:54.206396 2559 generic.go:356] "Generic (PLEG): container finished" podID="15d629b2-d3a1-4423-8df3-39c6661b4a71" containerID="555a004006f42578b80c924da4e43688357af90150405f6b26af40bef62edfa3" exitCode=0 Jul 22 02:16:54.206481 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:54.206472 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" event={"ID":"15d629b2-d3a1-4423-8df3-39c6661b4a71","Type":"ContainerDied","Data":"555a004006f42578b80c924da4e43688357af90150405f6b26af40bef62edfa3"} Jul 22 02:16:54.206963 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:54.206945 2559 scope.go:117] "RemoveContainer" containerID="555a004006f42578b80c924da4e43688357af90150405f6b26af40bef62edfa3" Jul 22 02:16:54.208027 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:54.207999 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:16:54.208087 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:54.208068 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:16:54.208125 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:54.208102 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:16:55.213750 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:55.213714 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" event={"ID":"15d629b2-d3a1-4423-8df3-39c6661b4a71","Type":"ContainerStarted","Data":"b5b653c16902d80032953204e32dd6ac683490c58c28586de802aff65f7ac2d5"} Jul 22 02:16:55.591110 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:55.591079 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:55.591475 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:55.591452 2559 patch_prober.go:28] interesting pod/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 22 02:16:55.591530 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:55.591509 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" podUID="15d629b2-d3a1-4423-8df3-39c6661b4a71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 22 02:16:55.706321 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:55.706288 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 02:16:55.709740 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:55.709707 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 02:16:55.710997 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:55.710973 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 02:16:55.714302 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:55.714282 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 02:16:56.590921 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:56.590875 2559 patch_prober.go:28] interesting pod/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 22 02:16:56.591505 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:56.590942 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" podUID="15d629b2-d3a1-4423-8df3-39c6661b4a71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 22 02:16:57.591430 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:57.591394 2559 patch_prober.go:28] interesting pod/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 22 02:16:57.591874 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:57.591459 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" podUID="15d629b2-d3a1-4423-8df3-39c6661b4a71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 22 02:16:58.591274 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:58.591237 2559 patch_prober.go:28] interesting pod/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 22 02:16:58.591465 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:58.591317 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" podUID="15d629b2-d3a1-4423-8df3-39c6661b4a71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 22 02:16:59.595520 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:59.595490 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:59.595971 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:59.595903 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:16:59.596693 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:16:59.596675 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:17:01.164308 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:01.164272 2559 patch_prober.go:28] interesting pod/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 22 02:17:01.164782 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:01.164377 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" podUID="7ee7c7da-a2e5-4f23-a427-f2246f721b02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 22 02:17:01.249000 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:01.248966 2559 generic.go:356] "Generic (PLEG): container finished" podID="7ee7c7da-a2e5-4f23-a427-f2246f721b02" containerID="e005dfd9ec244c6493a6d8232afece96c81cf268d6b9a58c1bd3ffc8027de8ec" exitCode=0 Jul 22 02:17:01.249170 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:01.249040 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" event={"ID":"7ee7c7da-a2e5-4f23-a427-f2246f721b02","Type":"ContainerDied","Data":"e005dfd9ec244c6493a6d8232afece96c81cf268d6b9a58c1bd3ffc8027de8ec"} Jul 22 02:17:01.249551 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:01.249535 2559 scope.go:117] "RemoveContainer" containerID="e005dfd9ec244c6493a6d8232afece96c81cf268d6b9a58c1bd3ffc8027de8ec" Jul 22 02:17:01.250515 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:01.250487 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:17:01.250583 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:01.250553 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:17:01.250628 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:01.250584 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 22 02:17:02.258723 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:02.258682 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" event={"ID":"7ee7c7da-a2e5-4f23-a427-f2246f721b02","Type":"ContainerStarted","Data":"803e66a85b261384220e8a48c66e237c9ec1ccc95a5104ba3566aeacd304a92f"} Jul 22 02:17:02.987002 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:02.986967 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:17:02.987176 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:02.987125 2559 patch_prober.go:28] interesting pod/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 22 02:17:02.987227 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:02.987174 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" podUID="7ee7c7da-a2e5-4f23-a427-f2246f721b02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 22 02:17:03.987165 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:03.987122 2559 patch_prober.go:28] interesting pod/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 22 02:17:03.987678 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:03.987194 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" podUID="7ee7c7da-a2e5-4f23-a427-f2246f721b02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 22 02:17:04.987744 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:04.987703 2559 patch_prober.go:28] interesting pod/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 22 02:17:04.988134 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:04.987763 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" podUID="7ee7c7da-a2e5-4f23-a427-f2246f721b02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 22 02:17:05.987708 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:05.987665 2559 patch_prober.go:28] interesting pod/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 22 02:17:05.987925 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:05.987728 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" podUID="7ee7c7da-a2e5-4f23-a427-f2246f721b02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 22 02:17:06.991134 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:06.991100 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:17:06.991755 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:06.991535 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:17:06.992147 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:17:06.992127 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:21:55.737988 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:21:55.737906 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 02:21:55.743317 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:21:55.743289 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 02:21:55.743770 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:21:55.743735 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 02:21:55.748553 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:21:55.748535 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7gfnj_72d8f397-d16d-446b-9d87-c2064ba99f21/ovn-acl-logging/0.log" Jul 22 02:21:55.976458 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:21:55.976428 2559 scope.go:117] "RemoveContainer" containerID="5f132e213812896d421ad4713901f2fe5807ba4dd053e8cb630fabf8d9bc6a21" Jul 22 02:23:03.535466 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:03.535429 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt"] Jul 22 02:23:03.536001 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:03.535665 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" podUID="f69a1a75-7dc8-435a-ad4d-1559b859cff9" containerName="maas-api" containerID="cri-o://7afa3cfd24042b49d1780385df9a69a2633b648a4b13ca341f3284ceeeaf7809" gracePeriod=30 Jul 22 02:23:03.803849 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:03.803820 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" Jul 22 02:23:03.935122 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:03.935089 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f69a1a75-7dc8-435a-ad4d-1559b859cff9-openshift-service-ca\") pod \"f69a1a75-7dc8-435a-ad4d-1559b859cff9\" (UID: \"f69a1a75-7dc8-435a-ad4d-1559b859cff9\") " Jul 22 02:23:03.935324 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:03.935145 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qj46s\" (UniqueName: \"kubernetes.io/projected/f69a1a75-7dc8-435a-ad4d-1559b859cff9-kube-api-access-qj46s\") pod \"f69a1a75-7dc8-435a-ad4d-1559b859cff9\" (UID: \"f69a1a75-7dc8-435a-ad4d-1559b859cff9\") " Jul 22 02:23:03.935324 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:03.935171 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f69a1a75-7dc8-435a-ad4d-1559b859cff9-maas-api-tls\") pod \"f69a1a75-7dc8-435a-ad4d-1559b859cff9\" (UID: \"f69a1a75-7dc8-435a-ad4d-1559b859cff9\") " Jul 22 02:23:03.935530 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:03.935495 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f69a1a75-7dc8-435a-ad4d-1559b859cff9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f69a1a75-7dc8-435a-ad4d-1559b859cff9" (UID: "f69a1a75-7dc8-435a-ad4d-1559b859cff9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:23:03.937270 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:03.937249 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f69a1a75-7dc8-435a-ad4d-1559b859cff9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f69a1a75-7dc8-435a-ad4d-1559b859cff9" (UID: "f69a1a75-7dc8-435a-ad4d-1559b859cff9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:23:03.937376 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:03.937334 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f69a1a75-7dc8-435a-ad4d-1559b859cff9-kube-api-access-qj46s" (OuterVolumeSpecName: "kube-api-access-qj46s") pod "f69a1a75-7dc8-435a-ad4d-1559b859cff9" (UID: "f69a1a75-7dc8-435a-ad4d-1559b859cff9"). InnerVolumeSpecName "kube-api-access-qj46s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:23:04.036052 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:04.036012 2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f69a1a75-7dc8-435a-ad4d-1559b859cff9-openshift-service-ca\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:04.036052 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:04.036044 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qj46s\" (UniqueName: \"kubernetes.io/projected/f69a1a75-7dc8-435a-ad4d-1559b859cff9-kube-api-access-qj46s\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:04.036052 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:04.036057 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f69a1a75-7dc8-435a-ad4d-1559b859cff9-maas-api-tls\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:04.048204 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:04.048118 2559 generic.go:356] "Generic (PLEG): container finished" podID="f69a1a75-7dc8-435a-ad4d-1559b859cff9" containerID="7afa3cfd24042b49d1780385df9a69a2633b648a4b13ca341f3284ceeeaf7809" exitCode=0 Jul 22 02:23:04.048204 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:04.048165 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" Jul 22 02:23:04.048440 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:04.048214 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" event={"ID":"f69a1a75-7dc8-435a-ad4d-1559b859cff9","Type":"ContainerDied","Data":"7afa3cfd24042b49d1780385df9a69a2633b648a4b13ca341f3284ceeeaf7809"} Jul 22 02:23:04.048440 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:04.048280 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt" event={"ID":"f69a1a75-7dc8-435a-ad4d-1559b859cff9","Type":"ContainerDied","Data":"35bb30097af5f1c0691fd96488d0efb4755a593d4dc6478cd362b512ffde51c5"} Jul 22 02:23:04.048440 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:04.048311 2559 scope.go:117] "RemoveContainer" containerID="7afa3cfd24042b49d1780385df9a69a2633b648a4b13ca341f3284ceeeaf7809" Jul 22 02:23:04.057283 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:04.057265 2559 scope.go:117] "RemoveContainer" containerID="7afa3cfd24042b49d1780385df9a69a2633b648a4b13ca341f3284ceeeaf7809" Jul 22 02:23:04.057557 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:23:04.057532 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7afa3cfd24042b49d1780385df9a69a2633b648a4b13ca341f3284ceeeaf7809\": container with ID starting with 7afa3cfd24042b49d1780385df9a69a2633b648a4b13ca341f3284ceeeaf7809 not found: ID does not exist" containerID="7afa3cfd24042b49d1780385df9a69a2633b648a4b13ca341f3284ceeeaf7809" Jul 22 02:23:04.057613 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:04.057566 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7afa3cfd24042b49d1780385df9a69a2633b648a4b13ca341f3284ceeeaf7809"} err="failed to get container status \"7afa3cfd24042b49d1780385df9a69a2633b648a4b13ca341f3284ceeeaf7809\": rpc error: code = NotFound desc = could not find container \"7afa3cfd24042b49d1780385df9a69a2633b648a4b13ca341f3284ceeeaf7809\": container with ID starting with 7afa3cfd24042b49d1780385df9a69a2633b648a4b13ca341f3284ceeeaf7809 not found: ID does not exist" Jul 22 02:23:04.070642 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:04.070621 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt"] Jul 22 02:23:04.074816 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:04.074794 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-eba9c8-596587dcf6-rx9nt"] Jul 22 02:23:05.626571 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:05.626541 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f69a1a75-7dc8-435a-ad4d-1559b859cff9" path="/var/lib/kubelet/pods/f69a1a75-7dc8-435a-ad4d-1559b859cff9/volumes" Jul 22 02:23:14.855835 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:14.855799 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v"] Jul 22 02:23:14.856311 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:14.856022 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" podUID="15d629b2-d3a1-4423-8df3-39c6661b4a71" containerName="istio-proxy" containerID="cri-o://b5b653c16902d80032953204e32dd6ac683490c58c28586de802aff65f7ac2d5" gracePeriod=30 Jul 22 02:23:20.112435 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.112401 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:23:20.133865 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.133780 2559 generic.go:356] "Generic (PLEG): container finished" podID="15d629b2-d3a1-4423-8df3-39c6661b4a71" containerID="b5b653c16902d80032953204e32dd6ac683490c58c28586de802aff65f7ac2d5" exitCode=0 Jul 22 02:23:20.134368 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.133846 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" Jul 22 02:23:20.134368 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.133875 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" event={"ID":"15d629b2-d3a1-4423-8df3-39c6661b4a71","Type":"ContainerDied","Data":"b5b653c16902d80032953204e32dd6ac683490c58c28586de802aff65f7ac2d5"} Jul 22 02:23:20.134368 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.134316 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v" event={"ID":"15d629b2-d3a1-4423-8df3-39c6661b4a71","Type":"ContainerDied","Data":"b1315fd9050fa5d1e7bd533fd31307e270f4bd6c416bd03f47cf8060e6e2fecc"} Jul 22 02:23:20.134612 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.134367 2559 scope.go:117] "RemoveContainer" containerID="b5b653c16902d80032953204e32dd6ac683490c58c28586de802aff65f7ac2d5" Jul 22 02:23:20.144436 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.144411 2559 scope.go:117] "RemoveContainer" containerID="555a004006f42578b80c924da4e43688357af90150405f6b26af40bef62edfa3" Jul 22 02:23:20.153005 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.152986 2559 scope.go:117] "RemoveContainer" containerID="b5b653c16902d80032953204e32dd6ac683490c58c28586de802aff65f7ac2d5" Jul 22 02:23:20.153288 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:23:20.153264 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5b653c16902d80032953204e32dd6ac683490c58c28586de802aff65f7ac2d5\": container with ID starting with b5b653c16902d80032953204e32dd6ac683490c58c28586de802aff65f7ac2d5 not found: ID does not exist" containerID="b5b653c16902d80032953204e32dd6ac683490c58c28586de802aff65f7ac2d5" Jul 22 02:23:20.153409 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.153300 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5b653c16902d80032953204e32dd6ac683490c58c28586de802aff65f7ac2d5"} err="failed to get container status \"b5b653c16902d80032953204e32dd6ac683490c58c28586de802aff65f7ac2d5\": rpc error: code = NotFound desc = could not find container \"b5b653c16902d80032953204e32dd6ac683490c58c28586de802aff65f7ac2d5\": container with ID starting with b5b653c16902d80032953204e32dd6ac683490c58c28586de802aff65f7ac2d5 not found: ID does not exist" Jul 22 02:23:20.153409 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.153319 2559 scope.go:117] "RemoveContainer" containerID="555a004006f42578b80c924da4e43688357af90150405f6b26af40bef62edfa3" Jul 22 02:23:20.153672 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:23:20.153653 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"555a004006f42578b80c924da4e43688357af90150405f6b26af40bef62edfa3\": container with ID starting with 555a004006f42578b80c924da4e43688357af90150405f6b26af40bef62edfa3 not found: ID does not exist" containerID="555a004006f42578b80c924da4e43688357af90150405f6b26af40bef62edfa3" Jul 22 02:23:20.153745 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.153677 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"555a004006f42578b80c924da4e43688357af90150405f6b26af40bef62edfa3"} err="failed to get container status \"555a004006f42578b80c924da4e43688357af90150405f6b26af40bef62edfa3\": rpc error: code = NotFound desc = could not find container \"555a004006f42578b80c924da4e43688357af90150405f6b26af40bef62edfa3\": container with ID starting with 555a004006f42578b80c924da4e43688357af90150405f6b26af40bef62edfa3 not found: ID does not exist" Jul 22 02:23:20.196806 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.196771 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbtsx\" (UniqueName: \"kubernetes.io/projected/15d629b2-d3a1-4423-8df3-39c6661b4a71-kube-api-access-jbtsx\") pod \"15d629b2-d3a1-4423-8df3-39c6661b4a71\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " Jul 22 02:23:20.197000 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.196819 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-credential-socket\") pod \"15d629b2-d3a1-4423-8df3-39c6661b4a71\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " Jul 22 02:23:20.197000 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.196841 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-token\") pod \"15d629b2-d3a1-4423-8df3-39c6661b4a71\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " Jul 22 02:23:20.197000 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.196869 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-envoy\") pod \"15d629b2-d3a1-4423-8df3-39c6661b4a71\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " Jul 22 02:23:20.197000 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.196915 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-workload-certs\") pod \"15d629b2-d3a1-4423-8df3-39c6661b4a71\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " Jul 22 02:23:20.197000 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.196980 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15d629b2-d3a1-4423-8df3-39c6661b4a71-istiod-ca-cert\") pod \"15d629b2-d3a1-4423-8df3-39c6661b4a71\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " Jul 22 02:23:20.197305 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.197017 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-data\") pod \"15d629b2-d3a1-4423-8df3-39c6661b4a71\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " Jul 22 02:23:20.197305 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.197043 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-podinfo\") pod \"15d629b2-d3a1-4423-8df3-39c6661b4a71\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " Jul 22 02:23:20.197305 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.197085 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-workload-socket\") pod \"15d629b2-d3a1-4423-8df3-39c6661b4a71\" (UID: \"15d629b2-d3a1-4423-8df3-39c6661b4a71\") " Jul 22 02:23:20.197305 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.197292 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "15d629b2-d3a1-4423-8df3-39c6661b4a71" (UID: "15d629b2-d3a1-4423-8df3-39c6661b4a71"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:23:20.197551 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.197518 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "15d629b2-d3a1-4423-8df3-39c6661b4a71" (UID: "15d629b2-d3a1-4423-8df3-39c6661b4a71"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:23:20.197551 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.197523 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-workload-certs\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:20.197982 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.197894 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "15d629b2-d3a1-4423-8df3-39c6661b4a71" (UID: "15d629b2-d3a1-4423-8df3-39c6661b4a71"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:23:20.197982 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.197850 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15d629b2-d3a1-4423-8df3-39c6661b4a71-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "15d629b2-d3a1-4423-8df3-39c6661b4a71" (UID: "15d629b2-d3a1-4423-8df3-39c6661b4a71"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:23:20.197982 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.197953 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-data" (OuterVolumeSpecName: "istio-data") pod "15d629b2-d3a1-4423-8df3-39c6661b4a71" (UID: "15d629b2-d3a1-4423-8df3-39c6661b4a71"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:23:20.199161 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.199131 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-token" (OuterVolumeSpecName: "istio-token") pod "15d629b2-d3a1-4423-8df3-39c6661b4a71" (UID: "15d629b2-d3a1-4423-8df3-39c6661b4a71"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:23:20.199249 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.199195 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15d629b2-d3a1-4423-8df3-39c6661b4a71-kube-api-access-jbtsx" (OuterVolumeSpecName: "kube-api-access-jbtsx") pod "15d629b2-d3a1-4423-8df3-39c6661b4a71" (UID: "15d629b2-d3a1-4423-8df3-39c6661b4a71"). InnerVolumeSpecName "kube-api-access-jbtsx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:23:20.199511 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.199488 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "15d629b2-d3a1-4423-8df3-39c6661b4a71" (UID: "15d629b2-d3a1-4423-8df3-39c6661b4a71"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:23:20.199589 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.199547 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "15d629b2-d3a1-4423-8df3-39c6661b4a71" (UID: "15d629b2-d3a1-4423-8df3-39c6661b4a71"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:23:20.298614 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.298570 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-data\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:20.298614 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.298610 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-podinfo\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:20.298614 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.298619 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-workload-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:20.298614 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.298627 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbtsx\" (UniqueName: \"kubernetes.io/projected/15d629b2-d3a1-4423-8df3-39c6661b4a71-kube-api-access-jbtsx\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:20.298614 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.298637 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-credential-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:20.298936 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.298645 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-token\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:20.298936 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.298653 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15d629b2-d3a1-4423-8df3-39c6661b4a71-istio-envoy\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:20.298936 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.298660 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15d629b2-d3a1-4423-8df3-39c6661b4a71-istiod-ca-cert\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:20.458467 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.458437 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v"] Jul 22 02:23:20.463580 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:20.463546 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-eba9c8-openshift-default-6d85695b75-brp9v"] Jul 22 02:23:21.626148 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:21.626116 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15d629b2-d3a1-4423-8df3-39c6661b4a71" path="/var/lib/kubelet/pods/15d629b2-d3a1-4423-8df3-39c6661b4a71/volumes" Jul 22 02:23:37.584039 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:37.583998 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w"] Jul 22 02:23:37.584674 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:37.584388 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" podUID="7ee7c7da-a2e5-4f23-a427-f2246f721b02" containerName="istio-proxy" containerID="cri-o://803e66a85b261384220e8a48c66e237c9ec1ccc95a5104ba3566aeacd304a92f" gracePeriod=30 Jul 22 02:23:42.833888 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.833858 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:23:42.930586 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.930485 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-workload-socket\") pod \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " Jul 22 02:23:42.930586 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.930557 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-data\") pod \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " Jul 22 02:23:42.930586 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.930579 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-envoy\") pod \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " Jul 22 02:23:42.930868 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.930663 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istiod-ca-cert\") pod \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " Jul 22 02:23:42.930868 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.930690 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-token\") pod \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " Jul 22 02:23:42.930868 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.930722 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-workload-certs\") pod \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " Jul 22 02:23:42.930868 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.930764 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-podinfo\") pod \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " Jul 22 02:23:42.930868 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.930799 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jw5pt\" (UniqueName: \"kubernetes.io/projected/7ee7c7da-a2e5-4f23-a427-f2246f721b02-kube-api-access-jw5pt\") pod \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " Jul 22 02:23:42.930868 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.930840 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-credential-socket\") pod \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\" (UID: \"7ee7c7da-a2e5-4f23-a427-f2246f721b02\") " Jul 22 02:23:42.930868 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.930835 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7ee7c7da-a2e5-4f23-a427-f2246f721b02" (UID: "7ee7c7da-a2e5-4f23-a427-f2246f721b02"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:23:42.931197 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.931064 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7ee7c7da-a2e5-4f23-a427-f2246f721b02" (UID: "7ee7c7da-a2e5-4f23-a427-f2246f721b02"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:23:42.931197 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.931183 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-workload-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:42.931313 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.931200 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istiod-ca-cert\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:42.931430 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.931308 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7ee7c7da-a2e5-4f23-a427-f2246f721b02" (UID: "7ee7c7da-a2e5-4f23-a427-f2246f721b02"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:23:42.931430 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.931325 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7ee7c7da-a2e5-4f23-a427-f2246f721b02" (UID: "7ee7c7da-a2e5-4f23-a427-f2246f721b02"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:23:42.931527 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.931439 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-data" (OuterVolumeSpecName: "istio-data") pod "7ee7c7da-a2e5-4f23-a427-f2246f721b02" (UID: "7ee7c7da-a2e5-4f23-a427-f2246f721b02"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:23:42.932981 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.932951 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7ee7c7da-a2e5-4f23-a427-f2246f721b02" (UID: "7ee7c7da-a2e5-4f23-a427-f2246f721b02"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:23:42.932981 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.932954 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7ee7c7da-a2e5-4f23-a427-f2246f721b02" (UID: "7ee7c7da-a2e5-4f23-a427-f2246f721b02"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 02:23:42.933138 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.932993 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-token" (OuterVolumeSpecName: "istio-token") pod "7ee7c7da-a2e5-4f23-a427-f2246f721b02" (UID: "7ee7c7da-a2e5-4f23-a427-f2246f721b02"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:23:42.933267 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:42.933249 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ee7c7da-a2e5-4f23-a427-f2246f721b02-kube-api-access-jw5pt" (OuterVolumeSpecName: "kube-api-access-jw5pt") pod "7ee7c7da-a2e5-4f23-a427-f2246f721b02" (UID: "7ee7c7da-a2e5-4f23-a427-f2246f721b02"). InnerVolumeSpecName "kube-api-access-jw5pt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:23:43.031985 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.031950 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-token\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:43.031985 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.031979 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-workload-certs\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:43.031985 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.031988 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-podinfo\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:43.032219 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.031997 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw5pt\" (UniqueName: \"kubernetes.io/projected/7ee7c7da-a2e5-4f23-a427-f2246f721b02-kube-api-access-jw5pt\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:43.032219 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.032007 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-credential-socket\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:43.032219 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.032015 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-data\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:43.032219 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.032023 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ee7c7da-a2e5-4f23-a427-f2246f721b02-istio-envoy\") on node \"ip-10-0-131-80.ec2.internal\" DevicePath \"\"" Jul 22 02:23:43.253513 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.253422 2559 generic.go:356] "Generic (PLEG): container finished" podID="7ee7c7da-a2e5-4f23-a427-f2246f721b02" containerID="803e66a85b261384220e8a48c66e237c9ec1ccc95a5104ba3566aeacd304a92f" exitCode=0 Jul 22 02:23:43.253513 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.253471 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" Jul 22 02:23:43.253513 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.253475 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" event={"ID":"7ee7c7da-a2e5-4f23-a427-f2246f721b02","Type":"ContainerDied","Data":"803e66a85b261384220e8a48c66e237c9ec1ccc95a5104ba3566aeacd304a92f"} Jul 22 02:23:43.253792 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.253530 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w" event={"ID":"7ee7c7da-a2e5-4f23-a427-f2246f721b02","Type":"ContainerDied","Data":"498ff10f4eebb29b7360b6bdebfb87e8247449a2c2481a83596faa4ad1442b05"} Jul 22 02:23:43.253792 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.253558 2559 scope.go:117] "RemoveContainer" containerID="803e66a85b261384220e8a48c66e237c9ec1ccc95a5104ba3566aeacd304a92f" Jul 22 02:23:43.263006 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.262986 2559 scope.go:117] "RemoveContainer" containerID="e005dfd9ec244c6493a6d8232afece96c81cf268d6b9a58c1bd3ffc8027de8ec" Jul 22 02:23:43.270897 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.270878 2559 scope.go:117] "RemoveContainer" containerID="803e66a85b261384220e8a48c66e237c9ec1ccc95a5104ba3566aeacd304a92f" Jul 22 02:23:43.271150 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:23:43.271128 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"803e66a85b261384220e8a48c66e237c9ec1ccc95a5104ba3566aeacd304a92f\": container with ID starting with 803e66a85b261384220e8a48c66e237c9ec1ccc95a5104ba3566aeacd304a92f not found: ID does not exist" containerID="803e66a85b261384220e8a48c66e237c9ec1ccc95a5104ba3566aeacd304a92f" Jul 22 02:23:43.271201 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.271158 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"803e66a85b261384220e8a48c66e237c9ec1ccc95a5104ba3566aeacd304a92f"} err="failed to get container status \"803e66a85b261384220e8a48c66e237c9ec1ccc95a5104ba3566aeacd304a92f\": rpc error: code = NotFound desc = could not find container \"803e66a85b261384220e8a48c66e237c9ec1ccc95a5104ba3566aeacd304a92f\": container with ID starting with 803e66a85b261384220e8a48c66e237c9ec1ccc95a5104ba3566aeacd304a92f not found: ID does not exist" Jul 22 02:23:43.271201 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.271176 2559 scope.go:117] "RemoveContainer" containerID="e005dfd9ec244c6493a6d8232afece96c81cf268d6b9a58c1bd3ffc8027de8ec" Jul 22 02:23:43.271442 ip-10-0-131-80 kubenswrapper[2559]: E0722 02:23:43.271423 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e005dfd9ec244c6493a6d8232afece96c81cf268d6b9a58c1bd3ffc8027de8ec\": container with ID starting with e005dfd9ec244c6493a6d8232afece96c81cf268d6b9a58c1bd3ffc8027de8ec not found: ID does not exist" containerID="e005dfd9ec244c6493a6d8232afece96c81cf268d6b9a58c1bd3ffc8027de8ec" Jul 22 02:23:43.271718 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.271448 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e005dfd9ec244c6493a6d8232afece96c81cf268d6b9a58c1bd3ffc8027de8ec"} err="failed to get container status \"e005dfd9ec244c6493a6d8232afece96c81cf268d6b9a58c1bd3ffc8027de8ec\": rpc error: code = NotFound desc = could not find container \"e005dfd9ec244c6493a6d8232afece96c81cf268d6b9a58c1bd3ffc8027de8ec\": container with ID starting with e005dfd9ec244c6493a6d8232afece96c81cf268d6b9a58c1bd3ffc8027de8ec not found: ID does not exist" Jul 22 02:23:43.279608 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.279105 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w"] Jul 22 02:23:43.281796 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.281760 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-c6960c-openshift-default-895878c-zmr6w"] Jul 22 02:23:43.625849 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:43.625808 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ee7c7da-a2e5-4f23-a427-f2246f721b02" path="/var/lib/kubelet/pods/7ee7c7da-a2e5-4f23-a427-f2246f721b02/volumes" Jul 22 02:23:45.369553 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:45.369528 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c578697c6-xf47b_07b210d8-f291-4212-83c5-b2e6398a55b5/copy-manifests/0.log" Jul 22 02:23:45.377517 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:45.377483 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c578697c6-xf47b_07b210d8-f291-4212-83c5-b2e6398a55b5/manager/0.log" Jul 22 02:23:45.890587 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:45.890547 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744760-7qpbv_7925c8d2-4273-4b98-bfd2-5005d27454c3/cleanup/0.log" Jul 22 02:23:49.031706 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:49.031676 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4sljsh_39722719-282b-48fe-be81-2472e3eaaada/istio-proxy/0.log" Jul 22 02:23:49.169454 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:49.169420 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5cw2t_8d6c76ff-846e-41aa-b84d-b1003ca82b6b/discovery/0.log" Jul 22 02:23:49.596860 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:49.596828 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wrrzl_9ccf5793-b290-4f72-a31d-a5d13b6dbd3e/istio-proxy/1.log" Jul 22 02:23:49.989114 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:49.989033 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-fb4497b6-psq7s_e694641c-026f-4a5f-918b-6fedb0b96cdd/router/0.log" Jul 22 02:23:50.409645 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:50.409617 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-xmjk7_df704e77-dd0a-4766-afcc-8aa69afc532f/main/0.log" Jul 22 02:23:50.805299 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:23:50.805212 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4qm8p_7f5131ff-c8f7-448c-b051-23dc404eff6b/main/0.log" Jul 22 02:24:00.558974 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:00.558943 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tlbhv_51b9bc4d-f596-4c94-9337-1428391e8aef/global-pull-secret-syncer/0.log" Jul 22 02:24:00.700404 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:00.700363 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xtb6t_49ed0d1f-ebe1-4552-978c-4386cc349f09/konnectivity-agent/0.log" Jul 22 02:24:00.775984 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:00.775956 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-80.ec2.internal_ae3d302a83e4b95708739b07d02ee6e8/haproxy/0.log" Jul 22 02:24:07.996878 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:07.996796 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-6f9lw_26c374ff-c41f-44fe-b7f4-6d9b92b6bcb1/cluster-monitoring-operator/0.log" Jul 22 02:24:08.045101 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:08.045071 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-h8mll_a58cd2c5-b36e-4ad1-aed7-f5b378012004/kube-state-metrics/0.log" Jul 22 02:24:08.098719 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:08.098682 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-h8mll_a58cd2c5-b36e-4ad1-aed7-f5b378012004/kube-rbac-proxy-main/0.log" Jul 22 02:24:08.159795 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:08.159759 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-h8mll_a58cd2c5-b36e-4ad1-aed7-f5b378012004/kube-rbac-proxy-self/0.log" Jul 22 02:24:08.466543 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:08.466514 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c9qzd_1acf37c9-d8f6-4dcf-b867-580ccc196908/node-exporter/0.log" Jul 22 02:24:08.515855 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:08.515828 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c9qzd_1acf37c9-d8f6-4dcf-b867-580ccc196908/kube-rbac-proxy/0.log" Jul 22 02:24:08.561652 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:08.561626 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c9qzd_1acf37c9-d8f6-4dcf-b867-580ccc196908/init-textfile/0.log" Jul 22 02:24:08.934440 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:08.934407 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f5546c5e-1ba0-4c67-a465-8f8e612129a1/prometheus/0.log" Jul 22 02:24:08.990786 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:08.990756 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f5546c5e-1ba0-4c67-a465-8f8e612129a1/config-reloader/0.log" Jul 22 02:24:09.051519 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:09.051489 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f5546c5e-1ba0-4c67-a465-8f8e612129a1/thanos-sidecar/0.log" Jul 22 02:24:09.095229 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:09.095198 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f5546c5e-1ba0-4c67-a465-8f8e612129a1/kube-rbac-proxy-web/0.log" Jul 22 02:24:09.129857 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:09.129831 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f5546c5e-1ba0-4c67-a465-8f8e612129a1/kube-rbac-proxy/0.log" Jul 22 02:24:09.165503 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:09.165482 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f5546c5e-1ba0-4c67-a465-8f8e612129a1/kube-rbac-proxy-thanos/0.log" Jul 22 02:24:09.210911 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:09.210833 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f5546c5e-1ba0-4c67-a465-8f8e612129a1/init-config-reloader/0.log" Jul 22 02:24:09.246686 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:09.246660 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-mbwwq_2f2bab6b-76d2-41b0-a2d9-8f8bfc824009/prometheus-operator/0.log" Jul 22 02:24:09.284851 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:09.284824 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-mbwwq_2f2bab6b-76d2-41b0-a2d9-8f8bfc824009/kube-rbac-proxy/0.log" Jul 22 02:24:10.096100 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.096065 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4"] Jul 22 02:24:10.096501 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.096450 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15d629b2-d3a1-4423-8df3-39c6661b4a71" containerName="istio-proxy" Jul 22 02:24:10.096501 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.096463 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="15d629b2-d3a1-4423-8df3-39c6661b4a71" containerName="istio-proxy" Jul 22 02:24:10.096501 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.096470 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ee7c7da-a2e5-4f23-a427-f2246f721b02" containerName="istio-proxy" Jul 22 02:24:10.096501 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.096476 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ee7c7da-a2e5-4f23-a427-f2246f721b02" containerName="istio-proxy" Jul 22 02:24:10.096501 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.096484 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ee7c7da-a2e5-4f23-a427-f2246f721b02" containerName="istio-proxy" Jul 22 02:24:10.096501 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.096491 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ee7c7da-a2e5-4f23-a427-f2246f721b02" containerName="istio-proxy" Jul 22 02:24:10.096501 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.096503 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f69a1a75-7dc8-435a-ad4d-1559b859cff9" containerName="maas-api" Jul 22 02:24:10.096734 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.096508 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f69a1a75-7dc8-435a-ad4d-1559b859cff9" containerName="maas-api" Jul 22 02:24:10.096734 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.096521 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15d629b2-d3a1-4423-8df3-39c6661b4a71" containerName="istio-proxy" Jul 22 02:24:10.096734 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.096538 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="15d629b2-d3a1-4423-8df3-39c6661b4a71" containerName="istio-proxy" Jul 22 02:24:10.096734 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.096596 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ee7c7da-a2e5-4f23-a427-f2246f721b02" containerName="istio-proxy" Jul 22 02:24:10.096734 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.096603 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ee7c7da-a2e5-4f23-a427-f2246f721b02" containerName="istio-proxy" Jul 22 02:24:10.096734 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.096611 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="15d629b2-d3a1-4423-8df3-39c6661b4a71" containerName="istio-proxy" Jul 22 02:24:10.096734 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.096617 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="15d629b2-d3a1-4423-8df3-39c6661b4a71" containerName="istio-proxy" Jul 22 02:24:10.096734 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.096625 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f69a1a75-7dc8-435a-ad4d-1559b859cff9" containerName="maas-api" Jul 22 02:24:10.100454 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.100423 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" Jul 22 02:24:10.104323 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.104272 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-knlbt\"/\"kube-root-ca.crt\"" Jul 22 02:24:10.104502 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.104422 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-knlbt\"/\"openshift-service-ca.crt\"" Jul 22 02:24:10.105265 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.105234 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-knlbt\"/\"default-dockercfg-9xmg5\"" Jul 22 02:24:10.116909 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.116885 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4"] Jul 22 02:24:10.195382 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.195313 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/11ab6edd-99ae-4e92-8d97-bdec18c1383b-proc\") pod \"perf-node-gather-daemonset-488v4\" (UID: \"11ab6edd-99ae-4e92-8d97-bdec18c1383b\") " pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" Jul 22 02:24:10.195552 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.195451 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/11ab6edd-99ae-4e92-8d97-bdec18c1383b-podres\") pod \"perf-node-gather-daemonset-488v4\" (UID: \"11ab6edd-99ae-4e92-8d97-bdec18c1383b\") " pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" Jul 22 02:24:10.195552 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.195492 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl25r\" (UniqueName: \"kubernetes.io/projected/11ab6edd-99ae-4e92-8d97-bdec18c1383b-kube-api-access-gl25r\") pod \"perf-node-gather-daemonset-488v4\" (UID: \"11ab6edd-99ae-4e92-8d97-bdec18c1383b\") " pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" Jul 22 02:24:10.195632 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.195577 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11ab6edd-99ae-4e92-8d97-bdec18c1383b-sys\") pod \"perf-node-gather-daemonset-488v4\" (UID: \"11ab6edd-99ae-4e92-8d97-bdec18c1383b\") " pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" Jul 22 02:24:10.195632 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.195611 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11ab6edd-99ae-4e92-8d97-bdec18c1383b-lib-modules\") pod \"perf-node-gather-daemonset-488v4\" (UID: \"11ab6edd-99ae-4e92-8d97-bdec18c1383b\") " pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" Jul 22 02:24:10.296706 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.296669 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/11ab6edd-99ae-4e92-8d97-bdec18c1383b-podres\") pod \"perf-node-gather-daemonset-488v4\" (UID: \"11ab6edd-99ae-4e92-8d97-bdec18c1383b\") " pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" Jul 22 02:24:10.296706 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.296717 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl25r\" (UniqueName: \"kubernetes.io/projected/11ab6edd-99ae-4e92-8d97-bdec18c1383b-kube-api-access-gl25r\") pod \"perf-node-gather-daemonset-488v4\" (UID: \"11ab6edd-99ae-4e92-8d97-bdec18c1383b\") " pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" Jul 22 02:24:10.296977 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.296758 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11ab6edd-99ae-4e92-8d97-bdec18c1383b-sys\") pod \"perf-node-gather-daemonset-488v4\" (UID: \"11ab6edd-99ae-4e92-8d97-bdec18c1383b\") " pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" Jul 22 02:24:10.296977 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.296774 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11ab6edd-99ae-4e92-8d97-bdec18c1383b-lib-modules\") pod \"perf-node-gather-daemonset-488v4\" (UID: \"11ab6edd-99ae-4e92-8d97-bdec18c1383b\") " pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" Jul 22 02:24:10.296977 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.296832 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/11ab6edd-99ae-4e92-8d97-bdec18c1383b-proc\") pod \"perf-node-gather-daemonset-488v4\" (UID: \"11ab6edd-99ae-4e92-8d97-bdec18c1383b\") " pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" Jul 22 02:24:10.296977 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.296885 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/11ab6edd-99ae-4e92-8d97-bdec18c1383b-podres\") pod \"perf-node-gather-daemonset-488v4\" (UID: \"11ab6edd-99ae-4e92-8d97-bdec18c1383b\") " pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" Jul 22 02:24:10.296977 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.296906 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/11ab6edd-99ae-4e92-8d97-bdec18c1383b-proc\") pod \"perf-node-gather-daemonset-488v4\" (UID: \"11ab6edd-99ae-4e92-8d97-bdec18c1383b\") " pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" Jul 22 02:24:10.296977 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.296900 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11ab6edd-99ae-4e92-8d97-bdec18c1383b-sys\") pod \"perf-node-gather-daemonset-488v4\" (UID: \"11ab6edd-99ae-4e92-8d97-bdec18c1383b\") " pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" Jul 22 02:24:10.297166 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.296974 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11ab6edd-99ae-4e92-8d97-bdec18c1383b-lib-modules\") pod \"perf-node-gather-daemonset-488v4\" (UID: \"11ab6edd-99ae-4e92-8d97-bdec18c1383b\") " pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" Jul 22 02:24:10.309478 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.309453 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl25r\" (UniqueName: \"kubernetes.io/projected/11ab6edd-99ae-4e92-8d97-bdec18c1383b-kube-api-access-gl25r\") pod \"perf-node-gather-daemonset-488v4\" (UID: \"11ab6edd-99ae-4e92-8d97-bdec18c1383b\") " pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" Jul 22 02:24:10.414057 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.413967 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" Jul 22 02:24:10.554608 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.554565 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4"] Jul 22 02:24:10.557576 ip-10-0-131-80 kubenswrapper[2559]: W0722 02:24:10.557546 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod11ab6edd_99ae_4e92_8d97_bdec18c1383b.slice/crio-3047b47deb77bbee463fb5dbac956c36db10d5b92e7738607a4a63b0cc8bceb0 WatchSource:0}: Error finding container 3047b47deb77bbee463fb5dbac956c36db10d5b92e7738607a4a63b0cc8bceb0: Status 404 returned error can't find the container with id 3047b47deb77bbee463fb5dbac956c36db10d5b92e7738607a4a63b0cc8bceb0 Jul 22 02:24:10.989980 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:10.989950 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-bmvq8_cc79c9c1-844a-4780-9aa1-b433eeea8848/networking-console-plugin/0.log" Jul 22 02:24:11.390292 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:11.390256 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" event={"ID":"11ab6edd-99ae-4e92-8d97-bdec18c1383b","Type":"ContainerStarted","Data":"ad32e4e6287b4d265409dc8ec1936f7a0d800b75633fe7624bbd1ebffd0e7b98"} Jul 22 02:24:11.390789 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:11.390310 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" event={"ID":"11ab6edd-99ae-4e92-8d97-bdec18c1383b","Type":"ContainerStarted","Data":"3047b47deb77bbee463fb5dbac956c36db10d5b92e7738607a4a63b0cc8bceb0"} Jul 22 02:24:11.390789 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:11.390387 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" Jul 22 02:24:11.410654 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:11.410605 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" podStartSLOduration=1.410588091 podStartE2EDuration="1.410588091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:24:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:24:11.408290105 +0000 UTC m=+2836.237538452" watchObservedRunningTime="2026-07-22 02:24:11.410588091 +0000 UTC m=+2836.239836464" Jul 22 02:24:11.620587 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:11.620553 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/1.log" Jul 22 02:24:11.625267 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:11.625242 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-z6wbm_546e33d1-6966-445c-9d8d-d10cf6631943/console-operator/2.log" Jul 22 02:24:12.116256 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:12.116213 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6cfb8c6644-9fc82_6ef1db5a-7aba-4df2-822a-495acb593e8b/console/0.log" Jul 22 02:24:12.789492 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:12.789458 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-68jls_01ab9b95-2508-4029-bd96-dcb5d29c5fb6/volume-data-source-validator/0.log" Jul 22 02:24:14.019502 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:14.019470 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g2zdj_cb80e90e-d908-490b-a9da-bb1d624d6615/dns/0.log" Jul 22 02:24:14.043843 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:14.043791 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g2zdj_cb80e90e-d908-490b-a9da-bb1d624d6615/kube-rbac-proxy/0.log" Jul 22 02:24:14.131764 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:14.131739 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zvt6m_acaa1746-a8a5-4517-9299-9a502758b7a4/dns-node-resolver/0.log" Jul 22 02:24:14.768557 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:14.768513 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rs2vg_e3459ca6-711d-4b8a-b1d1-3f65d03ce297/node-ca/0.log" Jul 22 02:24:15.793500 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:15.793469 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4sljsh_39722719-282b-48fe-be81-2472e3eaaada/istio-proxy/0.log" Jul 22 02:24:15.925698 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:15.925665 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5cw2t_8d6c76ff-846e-41aa-b84d-b1003ca82b6b/discovery/0.log" Jul 22 02:24:16.006235 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:16.006204 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wrrzl_9ccf5793-b290-4f72-a31d-a5d13b6dbd3e/istio-proxy/0.log" Jul 22 02:24:16.030167 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:16.030134 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wrrzl_9ccf5793-b290-4f72-a31d-a5d13b6dbd3e/istio-proxy/1.log" Jul 22 02:24:16.129471 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:16.129444 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-fb4497b6-psq7s_e694641c-026f-4a5f-918b-6fedb0b96cdd/router/0.log" Jul 22 02:24:16.876318 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:16.876273 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qhr55_0df0a233-5828-4368-96ca-134ab7641aed/serve-healthcheck-canary/0.log" Jul 22 02:24:17.405053 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:17.405027 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-knlbt/perf-node-gather-daemonset-488v4" Jul 22 02:24:17.470075 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:17.470042 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-flhhx_2f7b6603-7aa5-411a-8956-40e0982d98f4/insights-operator/0.log" Jul 22 02:24:17.470815 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:17.470792 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-flhhx_2f7b6603-7aa5-411a-8956-40e0982d98f4/insights-operator/1.log" Jul 22 02:24:17.495047 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:17.495019 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gdxvd_d6d3be9e-b32f-4f80-b49f-38854284252f/kube-rbac-proxy/0.log" Jul 22 02:24:17.522815 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:17.522785 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gdxvd_d6d3be9e-b32f-4f80-b49f-38854284252f/exporter/0.log" Jul 22 02:24:17.547968 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:17.547938 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gdxvd_d6d3be9e-b32f-4f80-b49f-38854284252f/extractor/0.log" Jul 22 02:24:20.559948 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:20.559870 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4sljsh_39722719-282b-48fe-be81-2472e3eaaada/istio-proxy/0.log" Jul 22 02:24:20.684051 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:20.684018 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5cw2t_8d6c76ff-846e-41aa-b84d-b1003ca82b6b/discovery/0.log" Jul 22 02:24:20.747521 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:20.747490 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wrrzl_9ccf5793-b290-4f72-a31d-a5d13b6dbd3e/istio-proxy/0.log" Jul 22 02:24:20.771889 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:20.771853 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wrrzl_9ccf5793-b290-4f72-a31d-a5d13b6dbd3e/istio-proxy/1.log" Jul 22 02:24:20.862078 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:20.862049 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-fb4497b6-psq7s_e694641c-026f-4a5f-918b-6fedb0b96cdd/router/0.log" Jul 22 02:24:20.953313 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:20.953283 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c578697c6-xf47b_07b210d8-f291-4212-83c5-b2e6398a55b5/manager/0.log" Jul 22 02:24:20.998374 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:20.998320 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c578697c6-xf47b_07b210d8-f291-4212-83c5-b2e6398a55b5/copy-manifests/0.log" Jul 22 02:24:22.084026 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:22.084000 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4sljsh_39722719-282b-48fe-be81-2472e3eaaada/istio-proxy/0.log" Jul 22 02:24:22.206865 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:22.206828 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5cw2t_8d6c76ff-846e-41aa-b84d-b1003ca82b6b/discovery/0.log" Jul 22 02:24:22.262617 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:22.262589 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wrrzl_9ccf5793-b290-4f72-a31d-a5d13b6dbd3e/istio-proxy/0.log" Jul 22 02:24:22.263599 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:22.263576 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-65486778df-42clz_f584425a-5e67-4b6b-82bb-67bfc7a8bdad/manager/0.log" Jul 22 02:24:22.288572 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:22.288547 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-wrrzl_9ccf5793-b290-4f72-a31d-a5d13b6dbd3e/istio-proxy/1.log" Jul 22 02:24:22.318993 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:22.318962 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-gxnhh_e09d8012-31ad-42cc-a89d-7eb482723777/openshift-lws-operator/0.log" Jul 22 02:24:22.359030 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:22.358960 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-fb4497b6-psq7s_e694641c-026f-4a5f-918b-6fedb0b96cdd/router/0.log" Jul 22 02:24:27.662615 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:27.662560 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-6jd8v_f721d6eb-f694-4e62-9133-12edd4f6c81c/kube-storage-version-migrator-operator/1.log" Jul 22 02:24:27.663365 ip-10-0-131-80 kubenswrapper[2559]: I0722 02:24:27.663324 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-6jd8v_f721d6eb-f694-4e62-9133-12edd4f6c81c/kube-storage-version-migrator-operator/0.log"