Jul 21 12:37:45.848781 ip-10-0-133-39 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 21 12:37:45.848794 ip-10-0-133-39 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 21 12:37:45.848801 ip-10-0-133-39 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 21 12:37:45.849027 ip-10-0-133-39 systemd[1]: Failed to start Kubernetes Kubelet. Jul 21 12:37:56.098074 ip-10-0-133-39 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 21 12:37:56.098097 ip-10-0-133-39 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 068ad56595c14340bfbd678c18bdab60 -- Jul 21 12:39:36.113067 ip-10-0-133-39 systemd[1]: Starting Kubernetes Kubelet... Jul 21 12:39:36.536803 ip-10-0-133-39 kubenswrapper[2555]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 12:39:36.536803 ip-10-0-133-39 kubenswrapper[2555]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 21 12:39:36.536803 ip-10-0-133-39 kubenswrapper[2555]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 12:39:36.536803 ip-10-0-133-39 kubenswrapper[2555]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 21 12:39:36.536803 ip-10-0-133-39 kubenswrapper[2555]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 12:39:36.538223 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.538132 2555 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 21 12:39:36.543225 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543204 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:39:36.543225 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543225 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:39:36.543304 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543229 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:39:36.543304 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543232 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:39:36.543304 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543235 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:39:36.543304 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543238 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:39:36.543304 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543240 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:39:36.543304 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543245 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:39:36.543304 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543247 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:39:36.543304 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543250 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:39:36.543304 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543252 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:39:36.543304 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543256 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:39:36.543304 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543259 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:39:36.543304 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543261 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:39:36.543304 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543265 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:39:36.543304 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543270 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:39:36.543304 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543273 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:39:36.543304 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543276 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:39:36.543304 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543279 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:39:36.543304 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543281 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:39:36.543304 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543284 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:39:36.543772 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543287 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:39:36.543772 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543290 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:39:36.543772 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543292 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:39:36.543772 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543295 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:39:36.543772 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543297 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:39:36.543772 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543300 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:39:36.543772 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543302 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:39:36.543772 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543305 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:39:36.543772 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543308 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:39:36.543772 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543310 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:39:36.543772 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543313 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:39:36.543772 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543316 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:39:36.543772 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543319 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:39:36.543772 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543321 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:39:36.543772 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543324 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:39:36.543772 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543326 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:39:36.543772 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543328 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:39:36.543772 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543331 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:39:36.543772 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543341 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:39:36.544234 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543343 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:39:36.544234 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543346 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:39:36.544234 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543348 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:39:36.544234 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543350 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:39:36.544234 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543353 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:39:36.544234 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543355 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:39:36.544234 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543358 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:39:36.544234 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543361 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:39:36.544234 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543363 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:39:36.544234 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543365 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:39:36.544234 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543368 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:39:36.544234 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543370 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:39:36.544234 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543373 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:39:36.544234 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543378 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:39:36.544234 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543382 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:39:36.544234 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543386 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:39:36.544234 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543389 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:39:36.544234 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543391 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:39:36.544234 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543394 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543397 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543400 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543402 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543405 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543407 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543410 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543412 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543414 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543417 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543419 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543422 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543424 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543427 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543431 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543436 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543439 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543441 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543444 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543447 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:39:36.544694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543449 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543452 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543454 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543457 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543459 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543462 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543465 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543467 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543471 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543474 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543476 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543479 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543482 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543484 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543486 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543489 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543491 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543494 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543496 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543499 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:39:36.545202 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.543501 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:39:36.545717 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544315 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:39:36.545717 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544323 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:39:36.545717 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544327 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:39:36.545717 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544330 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:39:36.545717 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544333 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:39:36.545717 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544335 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:39:36.545717 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544339 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:39:36.545717 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544342 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:39:36.545717 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544344 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:39:36.545717 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544347 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:39:36.545717 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544349 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:39:36.545717 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544352 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:39:36.545717 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544355 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:39:36.545717 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544358 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:39:36.545717 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544362 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:39:36.545717 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544365 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:39:36.545717 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544369 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:39:36.545717 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544373 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:39:36.545717 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544376 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:39:36.546240 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544379 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:39:36.546240 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544382 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:39:36.546240 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544385 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:39:36.546240 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544388 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:39:36.546240 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544391 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:39:36.546240 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544393 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:39:36.546240 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544396 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:39:36.546240 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544398 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:39:36.546240 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544401 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:39:36.546240 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544403 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:39:36.546240 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544406 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:39:36.546240 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544408 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:39:36.546240 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544411 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:39:36.546240 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544415 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:39:36.546240 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544418 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:39:36.546240 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544422 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:39:36.546240 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544424 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:39:36.546240 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544427 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:39:36.546240 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544430 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544432 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544435 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544439 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544441 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544445 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544449 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544451 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544454 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544456 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544458 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544461 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544464 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544466 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544469 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544472 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544475 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544477 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544480 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544483 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:39:36.546703 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544485 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544488 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544490 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544493 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544495 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544497 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544500 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544502 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544505 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544507 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544510 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544513 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544515 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544517 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544520 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544523 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544525 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544528 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544530 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544533 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:39:36.547211 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544535 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544537 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544540 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544543 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544545 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544548 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544551 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544553 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544556 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544559 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544561 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544563 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544566 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544568 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544570 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544573 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544575 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544577 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544580 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544582 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:39:36.547691 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544585 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.544587 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545000 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545005 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545010 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545013 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545016 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545019 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545022 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545024 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545027 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545029 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545032 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545034 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545037 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545039 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545042 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545045 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545048 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545051 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:39:36.548192 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545053 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545056 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545058 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545061 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545064 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545066 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545069 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545072 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545074 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545077 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545080 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545082 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545085 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545087 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545090 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545092 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545095 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545097 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545099 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545102 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:39:36.548663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545105 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545108 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545110 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545116 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545118 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545121 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545123 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545125 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545128 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545131 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545133 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545135 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545138 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545140 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545143 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545145 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545148 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545150 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545152 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545155 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:39:36.549165 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545157 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545160 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545162 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545164 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545167 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545169 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545172 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545174 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545176 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545179 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545182 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545184 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545187 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545190 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545192 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545197 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545200 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545202 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545204 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545207 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:39:36.549665 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545210 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:39:36.550164 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545214 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:39:36.550164 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545218 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:39:36.550164 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545221 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:39:36.550164 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545224 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:39:36.550164 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545228 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:39:36.550164 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545232 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:39:36.550164 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545235 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:39:36.550164 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545238 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:39:36.550164 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545241 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:39:36.550164 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545243 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:39:36.550164 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545246 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:39:36.550164 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545249 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:39:36.550164 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545251 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:39:36.550164 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545254 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:39:36.550164 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545256 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:39:36.550164 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545258 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:39:36.550164 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545261 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:39:36.550164 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545263 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545266 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545268 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545356 2555 flags.go:64] FLAG: --address="0.0.0.0" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545364 2555 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545372 2555 flags.go:64] FLAG: --anonymous-auth="true" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545376 2555 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545381 2555 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545385 2555 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545390 2555 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545395 2555 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545399 2555 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545402 2555 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545405 2555 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545408 2555 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545411 2555 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545414 2555 flags.go:64] FLAG: --cgroup-root="" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545417 2555 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545420 2555 flags.go:64] FLAG: --client-ca-file="" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545423 2555 flags.go:64] FLAG: --cloud-provider="external" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545426 2555 flags.go:64] FLAG: --cluster-dns="[]" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545430 2555 flags.go:64] FLAG: --cluster-domain="" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545432 2555 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 21 12:39:36.550593 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545435 2555 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545439 2555 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545442 2555 flags.go:64] FLAG: --container-log-max-files="5" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545446 2555 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545449 2555 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545452 2555 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545455 2555 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545458 2555 flags.go:64] FLAG: --contention-profiling="false" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545462 2555 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545464 2555 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545468 2555 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545471 2555 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545475 2555 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545478 2555 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545481 2555 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545484 2555 flags.go:64] FLAG: --enable-load-reader="false" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545487 2555 flags.go:64] FLAG: --enable-server="true" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545490 2555 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545496 2555 flags.go:64] FLAG: --event-burst="100" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545499 2555 flags.go:64] FLAG: --event-qps="50" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545502 2555 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545506 2555 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545508 2555 flags.go:64] FLAG: --eviction-hard="" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545512 2555 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 21 12:39:36.551210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545515 2555 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545518 2555 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545521 2555 flags.go:64] FLAG: --eviction-soft="" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545523 2555 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545526 2555 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545529 2555 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545532 2555 flags.go:64] FLAG: --experimental-mounter-path="" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545534 2555 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545537 2555 flags.go:64] FLAG: --fail-swap-on="true" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545541 2555 flags.go:64] FLAG: --feature-gates="" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545545 2555 flags.go:64] FLAG: --file-check-frequency="20s" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545548 2555 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545551 2555 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545554 2555 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545557 2555 flags.go:64] FLAG: --healthz-port="10248" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545560 2555 flags.go:64] FLAG: --help="false" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545563 2555 flags.go:64] FLAG: --hostname-override="ip-10-0-133-39.ec2.internal" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545566 2555 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545569 2555 flags.go:64] FLAG: --http-check-frequency="20s" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545572 2555 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545575 2555 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545578 2555 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545580 2555 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 21 12:39:36.551790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545583 2555 flags.go:64] FLAG: --image-service-endpoint="" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545586 2555 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545588 2555 flags.go:64] FLAG: --kube-api-burst="100" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545592 2555 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545597 2555 flags.go:64] FLAG: --kube-api-qps="50" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545600 2555 flags.go:64] FLAG: --kube-reserved="" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545603 2555 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545606 2555 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545609 2555 flags.go:64] FLAG: --kubelet-cgroups="" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545611 2555 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545614 2555 flags.go:64] FLAG: --lock-file="" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545617 2555 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545620 2555 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545623 2555 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545628 2555 flags.go:64] FLAG: --log-json-split-stream="false" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545631 2555 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545633 2555 flags.go:64] FLAG: --log-text-split-stream="false" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545636 2555 flags.go:64] FLAG: --logging-format="text" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545643 2555 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545646 2555 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545649 2555 flags.go:64] FLAG: --manifest-url="" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545652 2555 flags.go:64] FLAG: --manifest-url-header="" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545656 2555 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545659 2555 flags.go:64] FLAG: --max-open-files="1000000" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545664 2555 flags.go:64] FLAG: --max-pods="110" Jul 21 12:39:36.552399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545667 2555 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545670 2555 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545672 2555 flags.go:64] FLAG: --memory-manager-policy="None" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545675 2555 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545678 2555 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545681 2555 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545684 2555 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545691 2555 flags.go:64] FLAG: --node-status-max-images="50" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545693 2555 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545696 2555 flags.go:64] FLAG: --oom-score-adj="-999" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545700 2555 flags.go:64] FLAG: --pod-cidr="" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545703 2555 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545709 2555 flags.go:64] FLAG: --pod-manifest-path="" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545712 2555 flags.go:64] FLAG: --pod-max-pids="-1" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545715 2555 flags.go:64] FLAG: --pods-per-core="0" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545718 2555 flags.go:64] FLAG: --port="10250" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545721 2555 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545724 2555 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b143f7a995f07d32" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545727 2555 flags.go:64] FLAG: --qos-reserved="" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545730 2555 flags.go:64] FLAG: --read-only-port="10255" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545733 2555 flags.go:64] FLAG: --register-node="true" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545736 2555 flags.go:64] FLAG: --register-with-taints="" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545739 2555 flags.go:64] FLAG: --registry-burst="10" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545742 2555 flags.go:64] FLAG: --registry-qps="5" Jul 21 12:39:36.553291 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545745 2555 flags.go:64] FLAG: --reserved-cpus="" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545749 2555 flags.go:64] FLAG: --reserved-memory="" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545766 2555 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545769 2555 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545772 2555 flags.go:64] FLAG: --rotate-certificates="false" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545775 2555 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545778 2555 flags.go:64] FLAG: --runonce="false" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545781 2555 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545784 2555 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545786 2555 flags.go:64] FLAG: --seccomp-default="false" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545789 2555 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545792 2555 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545795 2555 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545798 2555 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545802 2555 flags.go:64] FLAG: --storage-driver-password="root" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545804 2555 flags.go:64] FLAG: --storage-driver-secure="false" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545807 2555 flags.go:64] FLAG: --storage-driver-table="stats" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545810 2555 flags.go:64] FLAG: --storage-driver-user="root" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545813 2555 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545817 2555 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545819 2555 flags.go:64] FLAG: --system-cgroups="" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545823 2555 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545828 2555 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545831 2555 flags.go:64] FLAG: --tls-cert-file="" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545833 2555 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 21 12:39:36.554181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545837 2555 flags.go:64] FLAG: --tls-min-version="" Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545839 2555 flags.go:64] FLAG: --tls-private-key-file="" Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545842 2555 flags.go:64] FLAG: --topology-manager-policy="none" Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545845 2555 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545848 2555 flags.go:64] FLAG: --topology-manager-scope="container" Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545850 2555 flags.go:64] FLAG: --v="2" Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545855 2555 flags.go:64] FLAG: --version="false" Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545859 2555 flags.go:64] FLAG: --vmodule="" Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545864 2555 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.545867 2555 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545986 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545990 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545992 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545995 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.545998 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546001 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546003 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546006 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546008 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546010 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546013 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546015 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:39:36.554984 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546018 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546020 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546022 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546025 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546028 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546030 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546033 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546036 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546038 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546041 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546043 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546045 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546048 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546050 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546052 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546055 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546057 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546060 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546063 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546066 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:39:36.555550 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546068 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:39:36.556068 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546070 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:39:36.556068 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546073 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:39:36.556068 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546075 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:39:36.556068 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546078 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:39:36.556068 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546080 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:39:36.556068 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546082 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:39:36.556068 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546085 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:39:36.556068 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546087 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:39:36.556068 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546090 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:39:36.556068 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546092 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:39:36.556068 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546094 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:39:36.556068 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546097 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:39:36.556068 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546099 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:39:36.556068 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546102 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:39:36.556068 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546104 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:39:36.556068 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546107 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:39:36.556068 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546111 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:39:36.556068 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546114 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:39:36.556068 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546117 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:39:36.556537 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546120 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:39:36.556537 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546123 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:39:36.556537 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546126 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:39:36.556537 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546128 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:39:36.556537 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546131 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:39:36.556537 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546134 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:39:36.556537 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546137 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:39:36.556537 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546140 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:39:36.556537 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546143 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:39:36.556537 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546146 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:39:36.556537 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546150 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:39:36.556537 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546152 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:39:36.556537 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546155 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:39:36.556537 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546157 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:39:36.556537 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546160 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:39:36.556537 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546162 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:39:36.556537 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546164 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:39:36.556537 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546167 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:39:36.556537 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546171 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:39:36.557019 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546174 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:39:36.557019 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546177 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:39:36.557019 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546180 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:39:36.557019 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546182 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:39:36.557019 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546185 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:39:36.557019 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546187 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:39:36.557019 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546190 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:39:36.557019 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546192 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:39:36.557019 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546194 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:39:36.557019 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546197 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:39:36.557019 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546199 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:39:36.557019 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546202 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:39:36.557019 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546204 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:39:36.557019 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546206 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:39:36.557019 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546210 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:39:36.557019 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546212 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:39:36.557019 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546214 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:39:36.557019 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546216 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:39:36.557019 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546219 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:39:36.557495 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546221 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:39:36.557495 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546223 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:39:36.557495 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546226 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:39:36.557495 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546228 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:39:36.557495 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546232 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:39:36.557495 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546235 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:39:36.557495 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546237 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:39:36.557495 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546240 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:39:36.557495 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.546242 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:39:36.557495 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.546857 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 12:39:36.557495 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.555746 2555 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 21 12:39:36.557495 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.555783 2555 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 21 12:39:36.557495 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.555804 2555 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 21 12:39:36.557495 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.555813 2555 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 21 12:39:36.557860 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555875 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:39:36.557860 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555880 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:39:36.557860 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555883 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:39:36.557860 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555886 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:39:36.557860 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555889 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:39:36.557860 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555894 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:39:36.557860 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555898 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:39:36.557860 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555901 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:39:36.557860 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555904 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:39:36.557860 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555907 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:39:36.557860 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555910 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:39:36.557860 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555913 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:39:36.557860 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555915 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:39:36.557860 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555918 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:39:36.557860 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555921 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:39:36.557860 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555925 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:39:36.557860 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555928 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:39:36.557860 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555931 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:39:36.557860 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555934 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:39:36.558323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555936 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:39:36.558323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555939 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:39:36.558323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555942 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:39:36.558323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555944 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:39:36.558323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555947 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:39:36.558323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555949 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:39:36.558323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555952 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:39:36.558323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555954 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:39:36.558323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555957 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:39:36.558323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555961 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:39:36.558323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555963 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:39:36.558323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555966 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:39:36.558323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555968 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:39:36.558323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555971 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:39:36.558323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555973 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:39:36.558323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555976 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:39:36.558323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555978 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:39:36.558323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555981 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:39:36.558323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555983 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555986 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555989 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555991 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555994 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555996 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.555999 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556001 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556004 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556006 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556009 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556012 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556014 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556017 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556019 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556022 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556025 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556027 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556030 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556032 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:39:36.558856 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556034 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556037 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556040 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556043 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556046 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556048 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556050 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556053 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556055 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556058 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556060 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556063 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556065 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556068 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556070 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556073 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556075 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556078 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556081 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556083 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:39:36.559421 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556086 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556088 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556097 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556100 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556102 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556106 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556109 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556112 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556115 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556118 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556120 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556123 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556127 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556129 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556132 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556134 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556137 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556139 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556142 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556144 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:39:36.559941 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556147 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:39:36.560459 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556150 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:39:36.560459 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.556155 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 12:39:36.560459 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556272 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 12:39:36.560459 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556277 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 12:39:36.560459 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556280 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 12:39:36.560459 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556283 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 12:39:36.560459 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556286 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 12:39:36.560459 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556288 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 12:39:36.560459 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556291 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 12:39:36.560459 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556293 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 12:39:36.560459 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556296 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 12:39:36.560459 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556298 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 12:39:36.560459 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556301 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 12:39:36.560459 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556304 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 12:39:36.560459 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556307 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 12:39:36.560862 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556309 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 12:39:36.560862 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556312 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 12:39:36.560862 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556314 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 12:39:36.560862 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556317 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 12:39:36.560862 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556319 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 12:39:36.560862 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556322 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 12:39:36.560862 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556324 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 12:39:36.560862 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556327 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 12:39:36.560862 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556330 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 12:39:36.560862 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556332 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 12:39:36.560862 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556335 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 12:39:36.560862 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556337 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 12:39:36.560862 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556340 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 12:39:36.560862 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556342 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 21 12:39:36.560862 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556345 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 12:39:36.560862 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556348 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 12:39:36.560862 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556350 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 12:39:36.560862 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556353 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 12:39:36.560862 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556355 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556358 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556360 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556363 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556365 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556367 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556370 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556372 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556375 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556377 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556379 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556382 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556386 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556389 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556393 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556395 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556398 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556400 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556403 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556405 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 12:39:36.561338 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556408 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556410 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556413 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556415 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556418 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556420 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556423 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556426 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556429 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556431 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556433 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556436 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556438 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556441 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556443 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556446 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556448 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556450 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556453 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556455 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 12:39:36.561834 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556457 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 12:39:36.562317 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556460 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 12:39:36.562317 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556462 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 12:39:36.562317 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556465 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 12:39:36.562317 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556467 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 12:39:36.562317 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556471 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 12:39:36.562317 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556474 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 12:39:36.562317 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556479 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 12:39:36.562317 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556482 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 12:39:36.562317 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556485 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 12:39:36.562317 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556488 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 12:39:36.562317 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556490 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 12:39:36.562317 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556493 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 12:39:36.562317 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556496 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 12:39:36.562317 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556499 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 12:39:36.562317 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556502 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 12:39:36.562317 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556505 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 12:39:36.562317 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556507 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 12:39:36.562317 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556510 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 12:39:36.562317 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556512 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 12:39:36.562955 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556514 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 12:39:36.562955 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556517 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 12:39:36.562955 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556519 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 12:39:36.562955 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556522 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 12:39:36.562955 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556524 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 12:39:36.562955 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556527 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 12:39:36.562955 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556529 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 12:39:36.562955 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:36.556531 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 12:39:36.562955 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.556536 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 12:39:36.562955 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.556686 2555 server.go:962] "Client rotation is on, will bootstrap in background" Jul 21 12:39:36.562955 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.558972 2555 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 21 12:39:36.562955 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.560570 2555 server.go:1019] "Starting client certificate rotation" Jul 21 12:39:36.562955 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.560660 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 12:39:36.562955 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.560694 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 12:39:36.563309 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.562859 2555 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 12:39:36.567505 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.567486 2555 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 12:39:36.578260 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.578232 2555 log.go:25] "Validated CRI v1 runtime API" Jul 21 12:39:36.580552 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.580537 2555 log.go:25] "Validated CRI v1 image API" Jul 21 12:39:36.581486 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.581471 2555 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 21 12:39:36.588647 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.588624 2555 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c5023c8c-6865-44b2-9875-587f2916379e:/dev/nvme0n1p3 f3f50265-5d91-46e4-93e8-5e12eab56ad8:/dev/nvme0n1p4] Jul 21 12:39:36.588710 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.588648 2555 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 21 12:39:36.595784 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.595744 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 12:39:36.596163 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.596055 2555 manager.go:216] Machine: {Timestamp:2026-07-21 12:39:36.594711747 +0000 UTC m=+0.366843452 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3117134 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2809d02e772c7c5caca86141dbc70c SystemUUID:ec2809d0-2e77-2c7c-5cac-a86141dbc70c BootID:068ad565-95c1-4340-bfbd-678c18bdab60 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f8:e4:49:8a:c1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f8:e4:49:8a:c1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:f5:f3:75:29:f9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 21 12:39:36.596163 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.596159 2555 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 21 12:39:36.596258 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.596250 2555 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 21 12:39:36.599902 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.599869 2555 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 21 12:39:36.600045 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.599905 2555 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-39.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 21 12:39:36.600627 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.600617 2555 topology_manager.go:138] "Creating topology manager with none policy" Jul 21 12:39:36.600660 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.600630 2555 container_manager_linux.go:309] "Creating device plugin manager" Jul 21 12:39:36.600660 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.600638 2555 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 12:39:36.601648 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.601635 2555 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 12:39:36.602887 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.602876 2555 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 21 12:39:36.606346 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.606332 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 21 12:39:36.606493 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.606483 2555 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 21 12:39:36.606527 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.606520 2555 kubelet.go:486] "Attempting to sync node with API server" Jul 21 12:39:36.606570 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.606535 2555 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 21 12:39:36.606570 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.606557 2555 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 21 12:39:36.606570 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.606569 2555 kubelet.go:398] "Adding apiserver pod source" Jul 21 12:39:36.606671 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.606582 2555 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 21 12:39:36.607117 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.607104 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 12:39:36.607184 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.607131 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 12:39:36.609300 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.609279 2555 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 21 12:39:36.611138 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.611122 2555 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 21 12:39:36.611204 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.611155 2555 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 21 12:39:36.612327 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.612313 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 21 12:39:36.612370 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.612337 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 21 12:39:36.612370 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.612347 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 21 12:39:36.612370 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.612356 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 21 12:39:36.612370 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.612365 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 21 12:39:36.612370 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.612372 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 21 12:39:36.612497 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.612380 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 21 12:39:36.612497 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.612386 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 21 12:39:36.612497 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.612393 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 21 12:39:36.612497 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.612399 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 21 12:39:36.612497 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.612408 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 21 12:39:36.612497 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.612414 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 21 12:39:36.613212 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.613201 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 21 12:39:36.613212 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.613212 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 21 12:39:36.615097 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.615077 2555 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-39.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 21 12:39:36.615178 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:36.615129 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 21 12:39:36.615260 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:36.615156 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-39.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 21 12:39:36.616738 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.616725 2555 server.go:1268] "Started kubelet" Jul 21 12:39:36.616886 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.616812 2555 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 21 12:39:36.617409 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.617373 2555 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 21 12:39:36.617495 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.617440 2555 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 21 12:39:36.617792 ip-10-0-133-39 systemd[1]: Started Kubernetes Kubelet. Jul 21 12:39:36.619073 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.619057 2555 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 21 12:39:36.619822 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.619807 2555 server.go:310] "Adding debug handlers to kubelet server" Jul 21 12:39:36.622452 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:36.621563 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-39.ec2.internal.18c44e6ba70d51a1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-39.ec2.internal,UID:ip-10-0-133-39.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-39.ec2.internal,},FirstTimestamp:2026-07-21 12:39:36.616694177 +0000 UTC m=+0.388825861,LastTimestamp:2026-07-21 12:39:36.616694177 +0000 UTC m=+0.388825861,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-39.ec2.internal,}" Jul 21 12:39:36.622452 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:36.622441 2555 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 21 12:39:36.625442 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.625426 2555 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 21 12:39:36.625582 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.625427 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 21 12:39:36.625582 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:36.625575 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-39.ec2.internal\" not found" Jul 21 12:39:36.626413 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.626385 2555 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 21 12:39:36.626511 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.626500 2555 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 21 12:39:36.626743 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.626728 2555 reconstruct.go:97] "Volume reconstruction finished" Jul 21 12:39:36.626851 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.626837 2555 reconciler.go:29] "Reconciler: start to sync state" Jul 21 12:39:36.627050 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.627029 2555 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 21 12:39:36.627689 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.627669 2555 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 21 12:39:36.627689 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.627690 2555 factory.go:55] Registering systemd factory Jul 21 12:39:36.627859 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.627700 2555 factory.go:223] Registration of the systemd container factory successfully Jul 21 12:39:36.628073 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.628058 2555 factory.go:153] Registering CRI-O factory Jul 21 12:39:36.628131 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.628079 2555 factory.go:223] Registration of the crio container factory successfully Jul 21 12:39:36.628131 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.628105 2555 factory.go:103] Registering Raw factory Jul 21 12:39:36.628220 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.628154 2555 manager.go:1196] Started watching for new ooms in manager Jul 21 12:39:36.628794 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.628779 2555 manager.go:349] Starting recovery of all containers Jul 21 12:39:36.639606 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.639419 2555 manager.go:354] Recovery completed Jul 21 12:39:36.641511 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:36.641477 2555 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 21 12:39:36.643186 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:36.643156 2555 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-39.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 21 12:39:36.643186 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:36.643168 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 21 12:39:36.644803 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.644786 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:39:36.646199 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.646172 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r52pg" Jul 21 12:39:36.646960 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.646941 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:39:36.647015 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.646983 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:39:36.647015 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.646999 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:39:36.647705 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.647693 2555 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 21 12:39:36.647705 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.647704 2555 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 21 12:39:36.647829 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.647720 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 21 12:39:36.647829 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.647696 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:39:36.648995 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.648984 2555 policy_none.go:49] "None policy: Start" Jul 21 12:39:36.649029 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.648999 2555 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 21 12:39:36.649029 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.649009 2555 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 21 12:39:36.649995 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.649984 2555 policy_none.go:47] "Start" Jul 21 12:39:36.650254 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.650240 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:39:36.650294 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.650270 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:39:36.650294 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.650283 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:39:36.655039 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.655021 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r52pg" Jul 21 12:39:36.660458 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:36.660383 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-39.ec2.internal.18c44e6ba8db30e4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-39.ec2.internal,UID:ip-10-0-133-39.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-39.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-39.ec2.internal,},FirstTimestamp:2026-07-21 12:39:36.646963428 +0000 UTC m=+0.419095116,LastTimestamp:2026-07-21 12:39:36.646963428 +0000 UTC m=+0.419095116,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-39.ec2.internal,}" Jul 21 12:39:36.679847 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.679826 2555 manager.go:337] "Starting Device Plugin manager" Jul 21 12:39:36.679977 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:36.679870 2555 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 21 12:39:36.679977 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.679882 2555 server.go:87] "Starting device plugin registration server" Jul 21 12:39:36.680171 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.680157 2555 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 21 12:39:36.680234 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.680173 2555 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 21 12:39:36.680283 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.680261 2555 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 21 12:39:36.680372 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.680360 2555 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 21 12:39:36.680407 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.680373 2555 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 21 12:39:36.681149 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:36.681120 2555 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 21 12:39:36.681241 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:36.681194 2555 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-39.ec2.internal\" not found" Jul 21 12:39:36.763957 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.763919 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 21 12:39:36.765386 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.765369 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 21 12:39:36.765519 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.765405 2555 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 21 12:39:36.765519 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.765431 2555 kubelet.go:2448] "Starting kubelet main sync loop" Jul 21 12:39:36.765519 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:36.765472 2555 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 21 12:39:36.768520 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.768432 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:39:36.781049 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.781028 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:39:36.781989 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.781969 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:39:36.782088 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.782000 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:39:36.782088 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.782011 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:39:36.782088 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.782035 2555 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-39.ec2.internal" Jul 21 12:39:36.792409 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.792345 2555 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-39.ec2.internal" Jul 21 12:39:36.792409 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:36.792375 2555 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-39.ec2.internal\": node \"ip-10-0-133-39.ec2.internal\" not found" Jul 21 12:39:36.832904 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:36.832874 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-39.ec2.internal\" not found" Jul 21 12:39:36.866561 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.866528 2555 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-39.ec2.internal"] Jul 21 12:39:36.866693 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.866605 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:39:36.868182 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.868161 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:39:36.868300 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.868198 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:39:36.868300 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.868213 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:39:36.869537 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.869524 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:39:36.869670 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.869653 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal" Jul 21 12:39:36.869774 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.869694 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:39:36.870672 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.870567 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:39:36.870672 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.870601 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:39:36.870672 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.870623 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:39:36.870854 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.870688 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:39:36.870854 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.870723 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:39:36.870854 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.870738 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:39:36.873289 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.873270 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-39.ec2.internal" Jul 21 12:39:36.873393 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.873299 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:39:36.874056 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.874038 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:39:36.874155 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.874065 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:39:36.874155 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.874075 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:39:36.904655 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:36.904624 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-39.ec2.internal\" not found" node="ip-10-0-133-39.ec2.internal" Jul 21 12:39:36.908952 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:36.908934 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-39.ec2.internal\" not found" node="ip-10-0-133-39.ec2.internal" Jul 21 12:39:36.929049 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.929011 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/630345abb3ba3c5c280743c1b41d6a29-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal\" (UID: \"630345abb3ba3c5c280743c1b41d6a29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal" Jul 21 12:39:36.929049 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.929049 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/630345abb3ba3c5c280743c1b41d6a29-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal\" (UID: \"630345abb3ba3c5c280743c1b41d6a29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal" Jul 21 12:39:36.929233 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:36.929069 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aa32358a6864982ffe270d580961f111-config\") pod \"kube-apiserver-proxy-ip-10-0-133-39.ec2.internal\" (UID: \"aa32358a6864982ffe270d580961f111\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-39.ec2.internal" Jul 21 12:39:36.933113 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:36.933094 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-39.ec2.internal\" not found" Jul 21 12:39:37.029379 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:37.029335 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/630345abb3ba3c5c280743c1b41d6a29-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal\" (UID: \"630345abb3ba3c5c280743c1b41d6a29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal" Jul 21 12:39:37.029379 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:37.029383 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/630345abb3ba3c5c280743c1b41d6a29-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal\" (UID: \"630345abb3ba3c5c280743c1b41d6a29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal" Jul 21 12:39:37.029647 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:37.029404 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aa32358a6864982ffe270d580961f111-config\") pod \"kube-apiserver-proxy-ip-10-0-133-39.ec2.internal\" (UID: \"aa32358a6864982ffe270d580961f111\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-39.ec2.internal" Jul 21 12:39:37.029647 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:37.029441 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aa32358a6864982ffe270d580961f111-config\") pod \"kube-apiserver-proxy-ip-10-0-133-39.ec2.internal\" (UID: \"aa32358a6864982ffe270d580961f111\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-39.ec2.internal" Jul 21 12:39:37.029647 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:37.029445 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/630345abb3ba3c5c280743c1b41d6a29-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal\" (UID: \"630345abb3ba3c5c280743c1b41d6a29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal" Jul 21 12:39:37.029647 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:37.029446 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/630345abb3ba3c5c280743c1b41d6a29-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal\" (UID: \"630345abb3ba3c5c280743c1b41d6a29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal" Jul 21 12:39:37.033429 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:37.033411 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-39.ec2.internal\" not found" Jul 21 12:39:37.134420 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:37.134341 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-39.ec2.internal\" not found" Jul 21 12:39:37.207574 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:37.207547 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal" Jul 21 12:39:37.212208 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:37.211481 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-39.ec2.internal" Jul 21 12:39:37.235357 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:37.235325 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-39.ec2.internal\" not found" Jul 21 12:39:37.335890 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:37.335833 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-39.ec2.internal\" not found" Jul 21 12:39:37.436458 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:37.436376 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-39.ec2.internal\" not found" Jul 21 12:39:37.536948 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:37.536918 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-39.ec2.internal\" not found" Jul 21 12:39:37.560137 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:37.560089 2555 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 21 12:39:37.560310 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:37.560289 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 12:39:37.637837 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:37.637805 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-39.ec2.internal\" not found" Jul 21 12:39:37.659089 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:37.659044 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 12:34:37 +0000 UTC" deadline="2028-04-05 01:14:35.57905004 +0000 UTC" Jul 21 12:39:37.659089 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:37.659078 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14964h34m57.919975366s" Jul 21 12:39:37.737904 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:37.737875 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-39.ec2.internal\" not found" Jul 21 12:39:37.737904 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:37.737909 2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:39:37.821227 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:37.821204 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 12:39:37.821392 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:37.821199 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa32358a6864982ffe270d580961f111.slice/crio-df96c331bd3b326d36a8646323920d2036e3df1028f6d6e0b10facbd145117e1 WatchSource:0}: Error finding container df96c331bd3b326d36a8646323920d2036e3df1028f6d6e0b10facbd145117e1: Status 404 returned error can't find the container with id df96c331bd3b326d36a8646323920d2036e3df1028f6d6e0b10facbd145117e1 Jul 21 12:39:37.821712 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:37.821694 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod630345abb3ba3c5c280743c1b41d6a29.slice/crio-359674b5fbca4eb08faac49b381c895f67127bffa1953814c583267fa6efbc0b WatchSource:0}: Error finding container 359674b5fbca4eb08faac49b381c895f67127bffa1953814c583267fa6efbc0b: Status 404 returned error can't find the container with id 359674b5fbca4eb08faac49b381c895f67127bffa1953814c583267fa6efbc0b Jul 21 12:39:37.822615 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:37.822595 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasSufficientMemory" Jul 21 12:39:37.822723 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:37.822634 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 12:39:37.822723 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:37.822651 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeHasSufficientPID" Jul 21 12:39:37.827406 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:37.827393 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 12:39:37.838095 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:37.838074 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-39.ec2.internal\" not found" Jul 21 12:39:37.938819 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:37.938785 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-39.ec2.internal\" not found" Jul 21 12:39:37.965257 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:37.965231 2555 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:39:38.027172 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:38.027103 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-39.ec2.internal" Jul 21 12:39:38.038436 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:38.038407 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 12:39:38.040315 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:38.040296 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal" Jul 21 12:39:38.054839 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:38.054813 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 12:39:39.364888 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.364851 2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:39:39.761823 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.761790 2555 apiserver.go:52] "Watching apiserver" Jul 21 12:39:39.767529 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.767499 2555 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 21 12:39:39.769294 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.769219 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal","openshift-multus/multus-dszxc","openshift-multus/network-metrics-daemon-z65kc","openshift-network-operator/iptables-alerter-7q8jz","openshift-ovn-kubernetes/ovnkube-node-ntfk8","kube-system/konnectivity-agent-x72xs","kube-system/kube-apiserver-proxy-ip-10-0-133-39.ec2.internal","openshift-cluster-node-tuning-operator/tuned-kxdvv","openshift-dns/node-resolver-chngz","openshift-image-registry/node-ca-v98xb","openshift-multus/multus-additional-cni-plugins-npbb8","openshift-network-diagnostics/network-check-target-jblzc","kube-system/global-pull-secret-syncer-vtvm8","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c"] Jul 21 12:39:39.771181 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.771147 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7q8jz" Jul 21 12:39:39.773200 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.773177 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.773656 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.773638 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 21 12:39:39.773863 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.773819 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 21 12:39:39.773995 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.773863 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 21 12:39:39.773995 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.773869 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2jhhq\"" Jul 21 12:39:39.775858 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.775838 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 21 12:39:39.775974 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.775871 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 21 12:39:39.776052 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.776034 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 21 12:39:39.776867 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.776710 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 21 12:39:39.776867 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.776769 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-stjnz\"" Jul 21 12:39:39.777215 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.777194 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:39.777305 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:39.777270 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z65kc" podUID="eb0efdf9-1311-4257-9f40-3c35bee29d66" Jul 21 12:39:39.784999 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.780853 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 21 12:39:39.784999 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.783098 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.784999 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.783635 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v98xb" Jul 21 12:39:39.786357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.786334 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 21 12:39:39.786605 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.786571 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 21 12:39:39.786703 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.786684 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 21 12:39:39.787159 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.787142 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 21 12:39:39.787399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.787384 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 21 12:39:39.787573 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.787553 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jh5w6\"" Jul 21 12:39:39.787690 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.787674 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 21 12:39:39.787786 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.787766 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 21 12:39:39.787922 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.787887 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 21 12:39:39.788093 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.788053 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 21 12:39:39.788498 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.788479 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x72xs" Jul 21 12:39:39.788716 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.788696 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cfl8l\"" Jul 21 12:39:39.790277 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.790259 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:39.792058 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.792039 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-chngz" Jul 21 12:39:39.794790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.794608 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 21 12:39:39.794879 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.794803 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 21 12:39:39.794953 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.794935 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 21 12:39:39.795783 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.795462 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xd589\"" Jul 21 12:39:39.797187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.795947 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 21 12:39:39.797187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.796119 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:39.797187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.796378 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 21 12:39:39.797187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.796765 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-var-lib-openvswitch\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.797187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.796799 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-node-log\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.797187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.796826 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nrrg\" (UniqueName: \"kubernetes.io/projected/313167cc-d4ec-4ab0-9f12-b66389ad21fb-kube-api-access-6nrrg\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.797187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.796849 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2e3b06f6-c94b-485a-8d0f-bf23abb58edd-iptables-alerter-script\") pod \"iptables-alerter-7q8jz\" (UID: \"2e3b06f6-c94b-485a-8d0f-bf23abb58edd\") " pod="openshift-network-operator/iptables-alerter-7q8jz" Jul 21 12:39:39.797187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.796872 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-cnibin\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.797187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.796894 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-os-release\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.797187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.796924 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-multus-socket-dir-parent\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.797187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.796946 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-host-run-k8s-cni-cncf-io\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.797187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.796968 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-systemd-units\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.797187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.796990 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-run-netns\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.797187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.796997 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 12:39:39.797187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797011 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-run-systemd\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.797187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797033 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-cni-netd\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.797187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797072 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2564902e-b620-4b49-9e45-43e1b7a43f87-cni-binary-copy\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.797187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797121 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2564902e-b620-4b49-9e45-43e1b7a43f87-multus-daemon-config\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.797187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797161 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-host-run-multus-certs\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.798525 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797186 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-etc-kubernetes\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.798525 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797210 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-etc-openvswitch\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.798525 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797232 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-run-ovn-kubernetes\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.798525 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797255 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-host-var-lib-cni-bin\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.798525 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797282 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-host-var-lib-cni-multus\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.798525 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797304 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/313167cc-d4ec-4ab0-9f12-b66389ad21fb-ovn-node-metrics-cert\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.798525 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797326 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/313167cc-d4ec-4ab0-9f12-b66389ad21fb-ovnkube-script-lib\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.798525 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797347 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8f4w\" (UniqueName: \"kubernetes.io/projected/ba7b78ab-59ae-485e-93cc-8005bd5b9a69-kube-api-access-z8f4w\") pod \"node-ca-v98xb\" (UID: \"ba7b78ab-59ae-485e-93cc-8005bd5b9a69\") " pod="openshift-image-registry/node-ca-v98xb" Jul 21 12:39:39.798525 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797372 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/313167cc-d4ec-4ab0-9f12-b66389ad21fb-ovnkube-config\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.798525 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797393 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-hostroot\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.798525 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797415 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sskk\" (UniqueName: \"kubernetes.io/projected/2564902e-b620-4b49-9e45-43e1b7a43f87-kube-api-access-4sskk\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.798525 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797439 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-multus-cni-dir\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.798525 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797480 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-host-run-netns\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.798525 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797506 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-multus-conf-dir\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.798525 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797536 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs\") pod \"network-metrics-daemon-z65kc\" (UID: \"eb0efdf9-1311-4257-9f40-3c35bee29d66\") " pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:39.798525 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797559 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvf4c\" (UniqueName: \"kubernetes.io/projected/eb0efdf9-1311-4257-9f40-3c35bee29d66-kube-api-access-tvf4c\") pod \"network-metrics-daemon-z65kc\" (UID: \"eb0efdf9-1311-4257-9f40-3c35bee29d66\") " pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:39.798525 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797654 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba7b78ab-59ae-485e-93cc-8005bd5b9a69-host\") pod \"node-ca-v98xb\" (UID: \"ba7b78ab-59ae-485e-93cc-8005bd5b9a69\") " pod="openshift-image-registry/node-ca-v98xb" Jul 21 12:39:39.799357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797680 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ba7b78ab-59ae-485e-93cc-8005bd5b9a69-serviceca\") pod \"node-ca-v98xb\" (UID: \"ba7b78ab-59ae-485e-93cc-8005bd5b9a69\") " pod="openshift-image-registry/node-ca-v98xb" Jul 21 12:39:39.799357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797716 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krw68\" (UniqueName: \"kubernetes.io/projected/2e3b06f6-c94b-485a-8d0f-bf23abb58edd-kube-api-access-krw68\") pod \"iptables-alerter-7q8jz\" (UID: \"2e3b06f6-c94b-485a-8d0f-bf23abb58edd\") " pod="openshift-network-operator/iptables-alerter-7q8jz" Jul 21 12:39:39.799357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797735 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 21 12:39:39.799357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.797738 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-host-var-lib-kubelet\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.799357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.798060 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vm5ss\"" Jul 21 12:39:39.799357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.798147 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 21 12:39:39.799357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.798446 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lw4cj\"" Jul 21 12:39:39.799357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.798578 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wx662\"" Jul 21 12:39:39.799357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.798696 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 21 12:39:39.799357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.798841 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-run-openvswitch\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.799357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.798894 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-log-socket\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.799357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.798918 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-cni-bin\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.799357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.798961 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.799357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.799006 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e3b06f6-c94b-485a-8d0f-bf23abb58edd-host-slash\") pod \"iptables-alerter-7q8jz\" (UID: \"2e3b06f6-c94b-485a-8d0f-bf23abb58edd\") " pod="openshift-network-operator/iptables-alerter-7q8jz" Jul 21 12:39:39.799357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.799031 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-system-cni-dir\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.799357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.799054 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-slash\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.799357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.799077 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-kubelet\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.799357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.799098 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-run-ovn\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.799357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.799120 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/313167cc-d4ec-4ab0-9f12-b66389ad21fb-env-overrides\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.800259 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.799567 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:39:39.800259 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:39.799672 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jblzc" podUID="2026fc18-7ca0-4d3e-8adf-aca525a12986" Jul 21 12:39:39.801745 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.801533 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:39.801745 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:39.801615 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtvm8" podUID="a807b465-26c5-46ad-8abb-09a4f7716bee" Jul 21 12:39:39.809923 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.803262 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:39.809923 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.805090 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 21 12:39:39.809923 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.805431 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tvsqj\"" Jul 21 12:39:39.809923 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.805681 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 21 12:39:39.809923 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.805851 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 21 12:39:39.815614 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.815569 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7fg8n" Jul 21 12:39:39.824572 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.824490 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7fg8n" Jul 21 12:39:39.882724 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.882690 2555 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 21 12:39:39.899770 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.899739 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-multus-socket-dir-parent\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.899949 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.899779 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-host-run-k8s-cni-cncf-io\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.899949 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.899808 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-host-run-multus-certs\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.899949 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.899837 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5348e68a-685f-42bb-bceb-2f325e562a66-system-cni-dir\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:39.899949 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.899863 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret\") pod \"global-pull-secret-syncer-vtvm8\" (UID: \"a807b465-26c5-46ad-8abb-09a4f7716bee\") " pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:39.899949 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.899888 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-systemd-units\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.899949 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.899896 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-host-run-k8s-cni-cncf-io\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.899949 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.899910 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-run-netns\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.899949 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.899935 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-run-systemd\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.900375 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.899960 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-host-run-multus-certs\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.900375 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.899975 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-run-netns\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.900375 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.899905 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-multus-socket-dir-parent\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.900375 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.899980 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-run-systemd\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.900375 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900012 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-systemd-units\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.900375 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900015 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-cni-netd\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.900375 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900047 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-cni-netd\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.900375 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900055 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-etc-kubernetes\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.900375 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900080 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-etc-openvswitch\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.900375 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900102 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-etc-kubernetes\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.900375 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900104 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-run-ovn-kubernetes\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.900375 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900140 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-etc-openvswitch\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.900375 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900146 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-host-var-lib-cni-bin\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.900375 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900173 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbe1e799-15a0-4706-a471-f2c8a2e61675-tmp\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:39.900375 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900175 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-run-ovn-kubernetes\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.900375 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900201 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5348e68a-685f-42bb-bceb-2f325e562a66-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:39.900375 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900225 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-host-var-lib-cni-bin\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.901223 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900227 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jj5l\" (UniqueName: \"kubernetes.io/projected/5348e68a-685f-42bb-bceb-2f325e562a66-kube-api-access-6jj5l\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:39.901223 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900254 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/313167cc-d4ec-4ab0-9f12-b66389ad21fb-ovn-node-metrics-cert\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.901223 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900278 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/313167cc-d4ec-4ab0-9f12-b66389ad21fb-ovnkube-config\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.901223 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900312 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5348e68a-685f-42bb-bceb-2f325e562a66-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:39.901223 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900335 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f54b837e-c8bb-4787-8667-4f860ffe6c5d-device-dir\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:39.901223 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900366 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-host-run-netns\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.901223 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900391 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-var-lib-kubelet\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:39.901223 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900413 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f54b837e-c8bb-4787-8667-4f860ffe6c5d-etc-selinux\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:39.901223 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900436 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs\") pod \"network-metrics-daemon-z65kc\" (UID: \"eb0efdf9-1311-4257-9f40-3c35bee29d66\") " pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:39.901223 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900459 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-host-var-lib-kubelet\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.901223 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900479 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-sys\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:39.901223 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900518 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58089888-e584-4b42-ae32-e8074321df87-tmp-dir\") pod \"node-resolver-chngz\" (UID: \"58089888-e584-4b42-ae32-e8074321df87\") " pod="openshift-dns/node-resolver-chngz" Jul 21 12:39:39.901223 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900541 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5348e68a-685f-42bb-bceb-2f325e562a66-os-release\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:39.901223 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900567 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-run-openvswitch\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.901223 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900607 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-log-socket\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.901223 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900635 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.901977 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900658 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-systemd\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:39.901977 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900680 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-run\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:39.901977 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900700 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-tuned\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:39.901977 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900712 2555 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 21 12:39:39.901977 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900726 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hxcx\" (UniqueName: \"kubernetes.io/projected/f54b837e-c8bb-4787-8667-4f860ffe6c5d-kube-api-access-5hxcx\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:39.901977 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900751 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/313167cc-d4ec-4ab0-9f12-b66389ad21fb-env-overrides\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.901977 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900774 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aaa90a3d-1518-43bb-bff2-319c194a66ea-agent-certs\") pod \"konnectivity-agent-x72xs\" (UID: \"aaa90a3d-1518-43bb-bff2-319c194a66ea\") " pod="kube-system/konnectivity-agent-x72xs" Jul 21 12:39:39.901977 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900799 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aaa90a3d-1518-43bb-bff2-319c194a66ea-konnectivity-ca\") pod \"konnectivity-agent-x72xs\" (UID: \"aaa90a3d-1518-43bb-bff2-319c194a66ea\") " pod="kube-system/konnectivity-agent-x72xs" Jul 21 12:39:39.901977 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900822 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-var-lib-openvswitch\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.901977 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900846 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-node-log\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.901977 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900873 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nrrg\" (UniqueName: \"kubernetes.io/projected/313167cc-d4ec-4ab0-9f12-b66389ad21fb-kube-api-access-6nrrg\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.901977 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900896 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2e3b06f6-c94b-485a-8d0f-bf23abb58edd-iptables-alerter-script\") pod \"iptables-alerter-7q8jz\" (UID: \"2e3b06f6-c94b-485a-8d0f-bf23abb58edd\") " pod="openshift-network-operator/iptables-alerter-7q8jz" Jul 21 12:39:39.901977 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900920 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-cnibin\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.901977 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900956 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-os-release\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.901977 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900980 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfhnr\" (UniqueName: \"kubernetes.io/projected/58089888-e584-4b42-ae32-e8074321df87-kube-api-access-bfhnr\") pod \"node-resolver-chngz\" (UID: \"58089888-e584-4b42-ae32-e8074321df87\") " pod="openshift-dns/node-resolver-chngz" Jul 21 12:39:39.901977 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.901008 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2564902e-b620-4b49-9e45-43e1b7a43f87-cni-binary-copy\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.901977 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.901032 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2564902e-b620-4b49-9e45-43e1b7a43f87-multus-daemon-config\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.902731 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.901053 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-sysctl-d\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:39.902731 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.901083 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-host-var-lib-cni-multus\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.902731 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.901105 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-host\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:39.902731 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.901150 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5348e68a-685f-42bb-bceb-2f325e562a66-cnibin\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:39.902731 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.901175 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5348e68a-685f-42bb-bceb-2f325e562a66-cni-binary-copy\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:39.902731 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.901205 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f54b837e-c8bb-4787-8667-4f860ffe6c5d-sys-fs\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:39.902731 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.901229 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/313167cc-d4ec-4ab0-9f12-b66389ad21fb-ovnkube-config\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.902731 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.901233 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/313167cc-d4ec-4ab0-9f12-b66389ad21fb-ovnkube-script-lib\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.902731 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.900719 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-host-var-lib-kubelet\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.902731 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.901564 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-run-openvswitch\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.902731 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.901618 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-log-socket\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.902731 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.901652 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.902731 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.901714 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/313167cc-d4ec-4ab0-9f12-b66389ad21fb-ovnkube-script-lib\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.902731 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.901765 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-var-lib-openvswitch\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.902731 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.901793 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-node-log\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.902731 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902022 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/313167cc-d4ec-4ab0-9f12-b66389ad21fb-env-overrides\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.902731 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902068 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-host-var-lib-cni-multus\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.903300 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902121 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-host-run-netns\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.903300 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:39.902197 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:39.903300 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:39.902265 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs podName:eb0efdf9-1311-4257-9f40-3c35bee29d66 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:40.402243699 +0000 UTC m=+3.020536630 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs") pod "network-metrics-daemon-z65kc" (UID: "eb0efdf9-1311-4257-9f40-3c35bee29d66") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:39.903300 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902381 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8f4w\" (UniqueName: \"kubernetes.io/projected/ba7b78ab-59ae-485e-93cc-8005bd5b9a69-kube-api-access-z8f4w\") pod \"node-ca-v98xb\" (UID: \"ba7b78ab-59ae-485e-93cc-8005bd5b9a69\") " pod="openshift-image-registry/node-ca-v98xb" Jul 21 12:39:39.903300 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902421 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-hostroot\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.903300 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902448 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2e3b06f6-c94b-485a-8d0f-bf23abb58edd-iptables-alerter-script\") pod \"iptables-alerter-7q8jz\" (UID: \"2e3b06f6-c94b-485a-8d0f-bf23abb58edd\") " pod="openshift-network-operator/iptables-alerter-7q8jz" Jul 21 12:39:39.903300 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902453 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-cnibin\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.903300 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902449 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sskk\" (UniqueName: \"kubernetes.io/projected/2564902e-b620-4b49-9e45-43e1b7a43f87-kube-api-access-4sskk\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.903300 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902513 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-hostroot\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.903300 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902618 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-os-release\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.903300 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902677 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-sysctl-conf\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:39.903300 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902711 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/58089888-e584-4b42-ae32-e8074321df87-hosts-file\") pod \"node-resolver-chngz\" (UID: \"58089888-e584-4b42-ae32-e8074321df87\") " pod="openshift-dns/node-resolver-chngz" Jul 21 12:39:39.903300 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902767 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f54b837e-c8bb-4787-8667-4f860ffe6c5d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:39.903300 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902793 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a807b465-26c5-46ad-8abb-09a4f7716bee-dbus\") pod \"global-pull-secret-syncer-vtvm8\" (UID: \"a807b465-26c5-46ad-8abb-09a4f7716bee\") " pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:39.903300 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902826 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-multus-cni-dir\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.903300 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902876 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-multus-conf-dir\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.903300 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902902 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-modprobe-d\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:39.904042 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902928 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lvlz\" (UniqueName: \"kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz\") pod \"network-check-target-jblzc\" (UID: \"2026fc18-7ca0-4d3e-8adf-aca525a12986\") " pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:39:39.904042 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902955 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvf4c\" (UniqueName: \"kubernetes.io/projected/eb0efdf9-1311-4257-9f40-3c35bee29d66-kube-api-access-tvf4c\") pod \"network-metrics-daemon-z65kc\" (UID: \"eb0efdf9-1311-4257-9f40-3c35bee29d66\") " pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:39.904042 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.902979 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba7b78ab-59ae-485e-93cc-8005bd5b9a69-host\") pod \"node-ca-v98xb\" (UID: \"ba7b78ab-59ae-485e-93cc-8005bd5b9a69\") " pod="openshift-image-registry/node-ca-v98xb" Jul 21 12:39:39.904042 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903002 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ba7b78ab-59ae-485e-93cc-8005bd5b9a69-serviceca\") pod \"node-ca-v98xb\" (UID: \"ba7b78ab-59ae-485e-93cc-8005bd5b9a69\") " pod="openshift-image-registry/node-ca-v98xb" Jul 21 12:39:39.904042 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903031 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2564902e-b620-4b49-9e45-43e1b7a43f87-cni-binary-copy\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.904042 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903090 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krw68\" (UniqueName: \"kubernetes.io/projected/2e3b06f6-c94b-485a-8d0f-bf23abb58edd-kube-api-access-krw68\") pod \"iptables-alerter-7q8jz\" (UID: \"2e3b06f6-c94b-485a-8d0f-bf23abb58edd\") " pod="openshift-network-operator/iptables-alerter-7q8jz" Jul 21 12:39:39.904042 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903118 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba7b78ab-59ae-485e-93cc-8005bd5b9a69-host\") pod \"node-ca-v98xb\" (UID: \"ba7b78ab-59ae-485e-93cc-8005bd5b9a69\") " pod="openshift-image-registry/node-ca-v98xb" Jul 21 12:39:39.904042 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903133 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-sysconfig\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:39.904042 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903151 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-lib-modules\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:39.904042 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903174 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-768tz\" (UniqueName: \"kubernetes.io/projected/fbe1e799-15a0-4706-a471-f2c8a2e61675-kube-api-access-768tz\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:39.904042 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903191 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5348e68a-685f-42bb-bceb-2f325e562a66-tuning-conf-dir\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:39.904042 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903091 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-multus-conf-dir\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.904042 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903232 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2564902e-b620-4b49-9e45-43e1b7a43f87-multus-daemon-config\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.904042 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903243 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-cni-bin\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.904042 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903264 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e3b06f6-c94b-485a-8d0f-bf23abb58edd-host-slash\") pod \"iptables-alerter-7q8jz\" (UID: \"2e3b06f6-c94b-485a-8d0f-bf23abb58edd\") " pod="openshift-network-operator/iptables-alerter-7q8jz" Jul 21 12:39:39.904042 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903280 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-system-cni-dir\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.904042 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903295 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-kubernetes\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:39.904725 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903335 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f54b837e-c8bb-4787-8667-4f860ffe6c5d-socket-dir\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:39.904725 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903382 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f54b837e-c8bb-4787-8667-4f860ffe6c5d-registration-dir\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:39.904725 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903400 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-slash\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.904725 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903416 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-kubelet\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.904725 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903444 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-run-ovn\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.904725 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903470 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a807b465-26c5-46ad-8abb-09a4f7716bee-kubelet-config\") pod \"global-pull-secret-syncer-vtvm8\" (UID: \"a807b465-26c5-46ad-8abb-09a4f7716bee\") " pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:39.904725 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903550 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-multus-cni-dir\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.904725 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903638 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-cni-bin\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.904725 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903674 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-slash\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.904725 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903681 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2e3b06f6-c94b-485a-8d0f-bf23abb58edd-host-slash\") pod \"iptables-alerter-7q8jz\" (UID: \"2e3b06f6-c94b-485a-8d0f-bf23abb58edd\") " pod="openshift-network-operator/iptables-alerter-7q8jz" Jul 21 12:39:39.904725 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903707 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-host-kubelet\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.904725 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903712 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ba7b78ab-59ae-485e-93cc-8005bd5b9a69-serviceca\") pod \"node-ca-v98xb\" (UID: \"ba7b78ab-59ae-485e-93cc-8005bd5b9a69\") " pod="openshift-image-registry/node-ca-v98xb" Jul 21 12:39:39.904725 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903720 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2564902e-b620-4b49-9e45-43e1b7a43f87-system-cni-dir\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.904725 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.903739 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/313167cc-d4ec-4ab0-9f12-b66389ad21fb-run-ovn\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.905485 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.905453 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/313167cc-d4ec-4ab0-9f12-b66389ad21fb-ovn-node-metrics-cert\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.914651 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.914306 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nrrg\" (UniqueName: \"kubernetes.io/projected/313167cc-d4ec-4ab0-9f12-b66389ad21fb-kube-api-access-6nrrg\") pod \"ovnkube-node-ntfk8\" (UID: \"313167cc-d4ec-4ab0-9f12-b66389ad21fb\") " pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:39.915215 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.915188 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvf4c\" (UniqueName: \"kubernetes.io/projected/eb0efdf9-1311-4257-9f40-3c35bee29d66-kube-api-access-tvf4c\") pod \"network-metrics-daemon-z65kc\" (UID: \"eb0efdf9-1311-4257-9f40-3c35bee29d66\") " pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:39.918263 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.918232 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krw68\" (UniqueName: \"kubernetes.io/projected/2e3b06f6-c94b-485a-8d0f-bf23abb58edd-kube-api-access-krw68\") pod \"iptables-alerter-7q8jz\" (UID: \"2e3b06f6-c94b-485a-8d0f-bf23abb58edd\") " pod="openshift-network-operator/iptables-alerter-7q8jz" Jul 21 12:39:39.918388 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.918241 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8f4w\" (UniqueName: \"kubernetes.io/projected/ba7b78ab-59ae-485e-93cc-8005bd5b9a69-kube-api-access-z8f4w\") pod \"node-ca-v98xb\" (UID: \"ba7b78ab-59ae-485e-93cc-8005bd5b9a69\") " pod="openshift-image-registry/node-ca-v98xb" Jul 21 12:39:39.921701 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.921671 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sskk\" (UniqueName: \"kubernetes.io/projected/2564902e-b620-4b49-9e45-43e1b7a43f87-kube-api-access-4sskk\") pod \"multus-dszxc\" (UID: \"2564902e-b620-4b49-9e45-43e1b7a43f87\") " pod="openshift-multus/multus-dszxc" Jul 21 12:39:39.924460 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.924408 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-39.ec2.internal" event={"ID":"aa32358a6864982ffe270d580961f111","Type":"ContainerStarted","Data":"df96c331bd3b326d36a8646323920d2036e3df1028f6d6e0b10facbd145117e1"} Jul 21 12:39:39.925538 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:39.925507 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal" event={"ID":"630345abb3ba3c5c280743c1b41d6a29","Type":"ContainerStarted","Data":"359674b5fbca4eb08faac49b381c895f67127bffa1953814c583267fa6efbc0b"} Jul 21 12:39:40.004418 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004376 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-kubernetes\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.004418 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004423 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f54b837e-c8bb-4787-8667-4f860ffe6c5d-socket-dir\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:40.004699 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004449 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f54b837e-c8bb-4787-8667-4f860ffe6c5d-registration-dir\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:40.004699 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004475 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a807b465-26c5-46ad-8abb-09a4f7716bee-kubelet-config\") pod \"global-pull-secret-syncer-vtvm8\" (UID: \"a807b465-26c5-46ad-8abb-09a4f7716bee\") " pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:40.004699 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004501 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5348e68a-685f-42bb-bceb-2f325e562a66-system-cni-dir\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:40.004699 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004532 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret\") pod \"global-pull-secret-syncer-vtvm8\" (UID: \"a807b465-26c5-46ad-8abb-09a4f7716bee\") " pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:40.004699 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004562 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbe1e799-15a0-4706-a471-f2c8a2e61675-tmp\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.004699 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004585 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5348e68a-685f-42bb-bceb-2f325e562a66-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:40.004699 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004632 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jj5l\" (UniqueName: \"kubernetes.io/projected/5348e68a-685f-42bb-bceb-2f325e562a66-kube-api-access-6jj5l\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:40.004699 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004659 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5348e68a-685f-42bb-bceb-2f325e562a66-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:40.004699 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004686 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f54b837e-c8bb-4787-8667-4f860ffe6c5d-device-dir\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:40.005029 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004710 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-var-lib-kubelet\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.005029 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004732 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f54b837e-c8bb-4787-8667-4f860ffe6c5d-etc-selinux\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:40.005029 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004770 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-sys\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.005029 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004792 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58089888-e584-4b42-ae32-e8074321df87-tmp-dir\") pod \"node-resolver-chngz\" (UID: \"58089888-e584-4b42-ae32-e8074321df87\") " pod="openshift-dns/node-resolver-chngz" Jul 21 12:39:40.005029 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004814 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5348e68a-685f-42bb-bceb-2f325e562a66-os-release\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:40.005029 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004840 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-systemd\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.005029 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004864 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-run\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.005029 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004885 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-tuned\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.005029 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004911 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hxcx\" (UniqueName: \"kubernetes.io/projected/f54b837e-c8bb-4787-8667-4f860ffe6c5d-kube-api-access-5hxcx\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:40.005029 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004937 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aaa90a3d-1518-43bb-bff2-319c194a66ea-agent-certs\") pod \"konnectivity-agent-x72xs\" (UID: \"aaa90a3d-1518-43bb-bff2-319c194a66ea\") " pod="kube-system/konnectivity-agent-x72xs" Jul 21 12:39:40.005029 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004962 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aaa90a3d-1518-43bb-bff2-319c194a66ea-konnectivity-ca\") pod \"konnectivity-agent-x72xs\" (UID: \"aaa90a3d-1518-43bb-bff2-319c194a66ea\") " pod="kube-system/konnectivity-agent-x72xs" Jul 21 12:39:40.005029 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.004996 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfhnr\" (UniqueName: \"kubernetes.io/projected/58089888-e584-4b42-ae32-e8074321df87-kube-api-access-bfhnr\") pod \"node-resolver-chngz\" (UID: \"58089888-e584-4b42-ae32-e8074321df87\") " pod="openshift-dns/node-resolver-chngz" Jul 21 12:39:40.005029 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005022 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-sysctl-d\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.005481 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005060 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-host\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.005481 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005085 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5348e68a-685f-42bb-bceb-2f325e562a66-cnibin\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:40.005481 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005111 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5348e68a-685f-42bb-bceb-2f325e562a66-cni-binary-copy\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:40.005481 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005134 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f54b837e-c8bb-4787-8667-4f860ffe6c5d-sys-fs\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:40.005481 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005162 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-sysctl-conf\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.005481 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005202 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/58089888-e584-4b42-ae32-e8074321df87-hosts-file\") pod \"node-resolver-chngz\" (UID: \"58089888-e584-4b42-ae32-e8074321df87\") " pod="openshift-dns/node-resolver-chngz" Jul 21 12:39:40.005481 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005227 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f54b837e-c8bb-4787-8667-4f860ffe6c5d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:40.005481 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005250 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a807b465-26c5-46ad-8abb-09a4f7716bee-dbus\") pod \"global-pull-secret-syncer-vtvm8\" (UID: \"a807b465-26c5-46ad-8abb-09a4f7716bee\") " pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:40.005481 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005277 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-modprobe-d\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.005481 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005302 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lvlz\" (UniqueName: \"kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz\") pod \"network-check-target-jblzc\" (UID: \"2026fc18-7ca0-4d3e-8adf-aca525a12986\") " pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:39:40.005481 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005329 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-sysconfig\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.005481 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005352 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-lib-modules\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.005481 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005375 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-768tz\" (UniqueName: \"kubernetes.io/projected/fbe1e799-15a0-4706-a471-f2c8a2e61675-kube-api-access-768tz\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.005481 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005400 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5348e68a-685f-42bb-bceb-2f325e562a66-tuning-conf-dir\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:40.005481 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005465 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5348e68a-685f-42bb-bceb-2f325e562a66-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:40.006034 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005527 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-kubernetes\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.006034 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005613 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f54b837e-c8bb-4787-8667-4f860ffe6c5d-socket-dir\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:40.006034 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005647 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5348e68a-685f-42bb-bceb-2f325e562a66-tuning-conf-dir\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:40.006034 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005663 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f54b837e-c8bb-4787-8667-4f860ffe6c5d-registration-dir\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:40.006034 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005707 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a807b465-26c5-46ad-8abb-09a4f7716bee-kubelet-config\") pod \"global-pull-secret-syncer-vtvm8\" (UID: \"a807b465-26c5-46ad-8abb-09a4f7716bee\") " pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:40.006034 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.005743 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5348e68a-685f-42bb-bceb-2f325e562a66-system-cni-dir\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:40.006034 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:40.005817 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:40.006034 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:40.005875 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret podName:a807b465-26c5-46ad-8abb-09a4f7716bee nodeName:}" failed. No retries permitted until 2026-07-21 12:39:40.505858441 +0000 UTC m=+3.124151370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret") pod "global-pull-secret-syncer-vtvm8" (UID: "a807b465-26c5-46ad-8abb-09a4f7716bee") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:40.006385 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.006360 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5348e68a-685f-42bb-bceb-2f325e562a66-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:40.006445 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.006436 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f54b837e-c8bb-4787-8667-4f860ffe6c5d-device-dir\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:40.006502 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.006487 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-var-lib-kubelet\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.006626 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.006609 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f54b837e-c8bb-4787-8667-4f860ffe6c5d-etc-selinux\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:40.006687 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.006668 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-sys\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.006945 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.006918 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58089888-e584-4b42-ae32-e8074321df87-tmp-dir\") pod \"node-resolver-chngz\" (UID: \"58089888-e584-4b42-ae32-e8074321df87\") " pod="openshift-dns/node-resolver-chngz" Jul 21 12:39:40.007072 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.007019 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5348e68a-685f-42bb-bceb-2f325e562a66-os-release\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:40.007145 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.007074 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-systemd\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.007145 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.007121 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-run\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.007582 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.007558 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f54b837e-c8bb-4787-8667-4f860ffe6c5d-sys-fs\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:40.007911 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.007860 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-sysctl-conf\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.007911 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.007886 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-host\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.008085 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.007925 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/58089888-e584-4b42-ae32-e8074321df87-hosts-file\") pod \"node-resolver-chngz\" (UID: \"58089888-e584-4b42-ae32-e8074321df87\") " pod="openshift-dns/node-resolver-chngz" Jul 21 12:39:40.008085 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.007925 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-sysctl-d\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.008085 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.007970 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f54b837e-c8bb-4787-8667-4f860ffe6c5d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:40.008085 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.008060 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a807b465-26c5-46ad-8abb-09a4f7716bee-dbus\") pod \"global-pull-secret-syncer-vtvm8\" (UID: \"a807b465-26c5-46ad-8abb-09a4f7716bee\") " pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:40.008204 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.008143 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-modprobe-d\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.008252 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.008236 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-lib-modules\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.008305 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.008291 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-sysconfig\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.008357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.007931 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5348e68a-685f-42bb-bceb-2f325e562a66-cnibin\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:40.008357 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.008309 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5348e68a-685f-42bb-bceb-2f325e562a66-cni-binary-copy\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:40.008841 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.008818 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/aaa90a3d-1518-43bb-bff2-319c194a66ea-konnectivity-ca\") pod \"konnectivity-agent-x72xs\" (UID: \"aaa90a3d-1518-43bb-bff2-319c194a66ea\") " pod="kube-system/konnectivity-agent-x72xs" Jul 21 12:39:40.009204 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.009163 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbe1e799-15a0-4706-a471-f2c8a2e61675-tmp\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.009477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.009457 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fbe1e799-15a0-4706-a471-f2c8a2e61675-etc-tuned\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.010712 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.010692 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/aaa90a3d-1518-43bb-bff2-319c194a66ea-agent-certs\") pod \"konnectivity-agent-x72xs\" (UID: \"aaa90a3d-1518-43bb-bff2-319c194a66ea\") " pod="kube-system/konnectivity-agent-x72xs" Jul 21 12:39:40.015151 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:40.015097 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:39:40.015151 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:40.015127 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:39:40.015151 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:40.015143 2555 projected.go:196] Error preparing data for projected volume kube-api-access-7lvlz for pod openshift-network-diagnostics/network-check-target-jblzc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:39:40.015405 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:40.015214 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz podName:2026fc18-7ca0-4d3e-8adf-aca525a12986 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:40.51519618 +0000 UTC m=+3.133489106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7lvlz" (UniqueName: "kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz") pod "network-check-target-jblzc" (UID: "2026fc18-7ca0-4d3e-8adf-aca525a12986") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:39:40.015671 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.015647 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jj5l\" (UniqueName: \"kubernetes.io/projected/5348e68a-685f-42bb-bceb-2f325e562a66-kube-api-access-6jj5l\") pod \"multus-additional-cni-plugins-npbb8\" (UID: \"5348e68a-685f-42bb-bceb-2f325e562a66\") " pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:40.016441 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.016418 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-768tz\" (UniqueName: \"kubernetes.io/projected/fbe1e799-15a0-4706-a471-f2c8a2e61675-kube-api-access-768tz\") pod \"tuned-kxdvv\" (UID: \"fbe1e799-15a0-4706-a471-f2c8a2e61675\") " pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.018614 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.017330 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfhnr\" (UniqueName: \"kubernetes.io/projected/58089888-e584-4b42-ae32-e8074321df87-kube-api-access-bfhnr\") pod \"node-resolver-chngz\" (UID: \"58089888-e584-4b42-ae32-e8074321df87\") " pod="openshift-dns/node-resolver-chngz" Jul 21 12:39:40.018614 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.017352 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hxcx\" (UniqueName: \"kubernetes.io/projected/f54b837e-c8bb-4787-8667-4f860ffe6c5d-kube-api-access-5hxcx\") pod \"aws-ebs-csi-driver-node-rb99c\" (UID: \"f54b837e-c8bb-4787-8667-4f860ffe6c5d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:40.071628 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.071575 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 12:39:40.089691 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.089642 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7q8jz" Jul 21 12:39:40.102948 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.102912 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dszxc" Jul 21 12:39:40.111284 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.111260 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x72xs" Jul 21 12:39:40.119196 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.119173 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:39:40.125936 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.125908 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v98xb" Jul 21 12:39:40.135060 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.135029 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" Jul 21 12:39:40.141185 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.141149 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-chngz" Jul 21 12:39:40.150451 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.150422 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-npbb8" Jul 21 12:39:40.156499 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.156469 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" Jul 21 12:39:40.408255 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.408158 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs\") pod \"network-metrics-daemon-z65kc\" (UID: \"eb0efdf9-1311-4257-9f40-3c35bee29d66\") " pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:40.408424 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:40.408341 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:40.408424 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:40.408419 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs podName:eb0efdf9-1311-4257-9f40-3c35bee29d66 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:41.408399297 +0000 UTC m=+4.026692238 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs") pod "network-metrics-daemon-z65kc" (UID: "eb0efdf9-1311-4257-9f40-3c35bee29d66") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:40.509466 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.509432 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret\") pod \"global-pull-secret-syncer-vtvm8\" (UID: \"a807b465-26c5-46ad-8abb-09a4f7716bee\") " pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:40.509689 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:40.509577 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:40.509689 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:40.509664 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret podName:a807b465-26c5-46ad-8abb-09a4f7716bee nodeName:}" failed. No retries permitted until 2026-07-21 12:39:41.509647001 +0000 UTC m=+4.127939940 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret") pod "global-pull-secret-syncer-vtvm8" (UID: "a807b465-26c5-46ad-8abb-09a4f7716bee") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:40.610634 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.610575 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lvlz\" (UniqueName: \"kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz\") pod \"network-check-target-jblzc\" (UID: \"2026fc18-7ca0-4d3e-8adf-aca525a12986\") " pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:39:40.610803 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:40.610733 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:39:40.610803 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:40.610759 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:39:40.610803 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:40.610774 2555 projected.go:196] Error preparing data for projected volume kube-api-access-7lvlz for pod openshift-network-diagnostics/network-check-target-jblzc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:39:40.610963 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:40.610833 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz podName:2026fc18-7ca0-4d3e-8adf-aca525a12986 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:41.610818053 +0000 UTC m=+4.229110982 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7lvlz" (UniqueName: "kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz") pod "network-check-target-jblzc" (UID: "2026fc18-7ca0-4d3e-8adf-aca525a12986") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:39:40.826875 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.826831 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 12:34:39 +0000 UTC" deadline="2028-05-06 07:55:01.245341274 +0000 UTC" Jul 21 12:39:40.827164 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.826877 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15715h15m20.418468289s" Jul 21 12:39:40.832392 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:40.832362 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaa90a3d_1518_43bb_bff2_319c194a66ea.slice/crio-2021b0e545031df8929de7a25732c57cc5298e09159f20c2ffb7c575ec62df62 WatchSource:0}: Error finding container 2021b0e545031df8929de7a25732c57cc5298e09159f20c2ffb7c575ec62df62: Status 404 returned error can't find the container with id 2021b0e545031df8929de7a25732c57cc5298e09159f20c2ffb7c575ec62df62 Jul 21 12:39:40.852761 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:40.852734 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58089888_e584_4b42_ae32_e8074321df87.slice/crio-a8f3a287e582e2bd727bfff5f7f548a4ff65fc16fe0a4ff13bcfd6f2b0f3db9f WatchSource:0}: Error finding container a8f3a287e582e2bd727bfff5f7f548a4ff65fc16fe0a4ff13bcfd6f2b0f3db9f: Status 404 returned error can't find the container with id a8f3a287e582e2bd727bfff5f7f548a4ff65fc16fe0a4ff13bcfd6f2b0f3db9f Jul 21 12:39:40.853423 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:40.853403 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf54b837e_c8bb_4787_8667_4f860ffe6c5d.slice/crio-c2f68df8b678ca3158dab86999384ac8f86d0501b752ecc41f6bf418f4155331 WatchSource:0}: Error finding container c2f68df8b678ca3158dab86999384ac8f86d0501b752ecc41f6bf418f4155331: Status 404 returned error can't find the container with id c2f68df8b678ca3158dab86999384ac8f86d0501b752ecc41f6bf418f4155331 Jul 21 12:39:40.854388 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:40.854362 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod313167cc_d4ec_4ab0_9f12_b66389ad21fb.slice/crio-c587525468d826745a5261fe2e2ff2dbde2acff4c53c5fe6ec7197ddf9acab2e WatchSource:0}: Error finding container c587525468d826745a5261fe2e2ff2dbde2acff4c53c5fe6ec7197ddf9acab2e: Status 404 returned error can't find the container with id c587525468d826745a5261fe2e2ff2dbde2acff4c53c5fe6ec7197ddf9acab2e Jul 21 12:39:40.856790 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:40.856733 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e3b06f6_c94b_485a_8d0f_bf23abb58edd.slice/crio-628e148545f9f0b73524648225ed97b428391658f657bc4d522fd2c10d5013be WatchSource:0}: Error finding container 628e148545f9f0b73524648225ed97b428391658f657bc4d522fd2c10d5013be: Status 404 returned error can't find the container with id 628e148545f9f0b73524648225ed97b428391658f657bc4d522fd2c10d5013be Jul 21 12:39:40.859509 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:40.859259 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba7b78ab_59ae_485e_93cc_8005bd5b9a69.slice/crio-f8faeba03aec328de078df81a43cf72b8388d6780fff1468a7d453d35041e148 WatchSource:0}: Error finding container f8faeba03aec328de078df81a43cf72b8388d6780fff1468a7d453d35041e148: Status 404 returned error can't find the container with id f8faeba03aec328de078df81a43cf72b8388d6780fff1468a7d453d35041e148 Jul 21 12:39:40.859752 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:39:40.859726 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2564902e_b620_4b49_9e45_43e1b7a43f87.slice/crio-f232d05327b2e8b0e0ce0e566513e5e97baf8a532153b6576f2415a64e454242 WatchSource:0}: Error finding container f232d05327b2e8b0e0ce0e566513e5e97baf8a532153b6576f2415a64e454242: Status 404 returned error can't find the container with id f232d05327b2e8b0e0ce0e566513e5e97baf8a532153b6576f2415a64e454242 Jul 21 12:39:40.929336 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.929153 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dszxc" event={"ID":"2564902e-b620-4b49-9e45-43e1b7a43f87","Type":"ContainerStarted","Data":"f232d05327b2e8b0e0ce0e566513e5e97baf8a532153b6576f2415a64e454242"} Jul 21 12:39:40.930332 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.930304 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v98xb" event={"ID":"ba7b78ab-59ae-485e-93cc-8005bd5b9a69","Type":"ContainerStarted","Data":"f8faeba03aec328de078df81a43cf72b8388d6780fff1468a7d453d35041e148"} Jul 21 12:39:40.932846 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.932782 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" event={"ID":"fbe1e799-15a0-4706-a471-f2c8a2e61675","Type":"ContainerStarted","Data":"393e0158c860ad1e01709880be5443fa487ae9031a79d721dc2d1392b4bc7a1c"} Jul 21 12:39:40.934459 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.934436 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x72xs" event={"ID":"aaa90a3d-1518-43bb-bff2-319c194a66ea","Type":"ContainerStarted","Data":"2021b0e545031df8929de7a25732c57cc5298e09159f20c2ffb7c575ec62df62"} Jul 21 12:39:40.936810 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.936789 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-npbb8" event={"ID":"5348e68a-685f-42bb-bceb-2f325e562a66","Type":"ContainerStarted","Data":"0db748e71cb90420d5bccae04f22c73a6fff3482a96e258ccbfc757972c0fa7e"} Jul 21 12:39:40.939309 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.939284 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7q8jz" event={"ID":"2e3b06f6-c94b-485a-8d0f-bf23abb58edd","Type":"ContainerStarted","Data":"628e148545f9f0b73524648225ed97b428391658f657bc4d522fd2c10d5013be"} Jul 21 12:39:40.940897 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.940875 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" event={"ID":"313167cc-d4ec-4ab0-9f12-b66389ad21fb","Type":"ContainerStarted","Data":"c587525468d826745a5261fe2e2ff2dbde2acff4c53c5fe6ec7197ddf9acab2e"} Jul 21 12:39:40.941833 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.941803 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" event={"ID":"f54b837e-c8bb-4787-8667-4f860ffe6c5d","Type":"ContainerStarted","Data":"c2f68df8b678ca3158dab86999384ac8f86d0501b752ecc41f6bf418f4155331"} Jul 21 12:39:40.944417 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:40.944397 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-chngz" event={"ID":"58089888-e584-4b42-ae32-e8074321df87","Type":"ContainerStarted","Data":"a8f3a287e582e2bd727bfff5f7f548a4ff65fc16fe0a4ff13bcfd6f2b0f3db9f"} Jul 21 12:39:41.424219 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:41.424181 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs\") pod \"network-metrics-daemon-z65kc\" (UID: \"eb0efdf9-1311-4257-9f40-3c35bee29d66\") " pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:41.424453 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:41.424433 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:41.424519 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:41.424509 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs podName:eb0efdf9-1311-4257-9f40-3c35bee29d66 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:43.42448877 +0000 UTC m=+6.042781717 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs") pod "network-metrics-daemon-z65kc" (UID: "eb0efdf9-1311-4257-9f40-3c35bee29d66") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:41.525190 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:41.525122 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret\") pod \"global-pull-secret-syncer-vtvm8\" (UID: \"a807b465-26c5-46ad-8abb-09a4f7716bee\") " pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:41.525381 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:41.525358 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:41.525448 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:41.525426 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret podName:a807b465-26c5-46ad-8abb-09a4f7716bee nodeName:}" failed. No retries permitted until 2026-07-21 12:39:43.525407703 +0000 UTC m=+6.143700649 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret") pod "global-pull-secret-syncer-vtvm8" (UID: "a807b465-26c5-46ad-8abb-09a4f7716bee") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:41.626413 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:41.626323 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lvlz\" (UniqueName: \"kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz\") pod \"network-check-target-jblzc\" (UID: \"2026fc18-7ca0-4d3e-8adf-aca525a12986\") " pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:39:41.626585 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:41.626507 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:39:41.626585 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:41.626526 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:39:41.626585 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:41.626540 2555 projected.go:196] Error preparing data for projected volume kube-api-access-7lvlz for pod openshift-network-diagnostics/network-check-target-jblzc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:39:41.626794 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:41.626615 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz podName:2026fc18-7ca0-4d3e-8adf-aca525a12986 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:43.626577359 +0000 UTC m=+6.244870300 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7lvlz" (UniqueName: "kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz") pod "network-check-target-jblzc" (UID: "2026fc18-7ca0-4d3e-8adf-aca525a12986") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:39:41.827227 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:41.827137 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 12:34:39 +0000 UTC" deadline="2028-01-17 22:07:06.559123087 +0000 UTC" Jul 21 12:39:41.827227 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:41.827190 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13089h27m24.731937759s" Jul 21 12:39:41.921452 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:41.920722 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:39:41.921452 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:41.920849 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jblzc" podUID="2026fc18-7ca0-4d3e-8adf-aca525a12986" Jul 21 12:39:41.921452 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:41.921241 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:41.921452 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:41.921338 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z65kc" podUID="eb0efdf9-1311-4257-9f40-3c35bee29d66" Jul 21 12:39:41.922022 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:41.921882 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:41.922022 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:41.921975 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtvm8" podUID="a807b465-26c5-46ad-8abb-09a4f7716bee" Jul 21 12:39:41.957729 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:41.957687 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-39.ec2.internal" event={"ID":"aa32358a6864982ffe270d580961f111","Type":"ContainerStarted","Data":"f04adaef8229543a8ee811de51792b49979431073cc3b5ce1994be17fa74dea7"} Jul 21 12:39:41.965824 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:41.965536 2555 generic.go:356] "Generic (PLEG): container finished" podID="630345abb3ba3c5c280743c1b41d6a29" containerID="a18c64300cf450af63b566d75a0db31a9aed2ebd55f067a95e31bfc6e07b8358" exitCode=0 Jul 21 12:39:41.965824 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:41.965587 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal" event={"ID":"630345abb3ba3c5c280743c1b41d6a29","Type":"ContainerDied","Data":"a18c64300cf450af63b566d75a0db31a9aed2ebd55f067a95e31bfc6e07b8358"} Jul 21 12:39:41.999124 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:41.999062 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-39.ec2.internal" podStartSLOduration=2.999037632 podStartE2EDuration="2.999037632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:39:41.979846888 +0000 UTC m=+4.598139838" watchObservedRunningTime="2026-07-21 12:39:41.999037632 +0000 UTC m=+4.617330581" Jul 21 12:39:42.993512 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:42.993474 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal" event={"ID":"630345abb3ba3c5c280743c1b41d6a29","Type":"ContainerStarted","Data":"809d29a232ddf5b14ef38364334cc31fc372601f0d375e0354aba15ea9d5948d"} Jul 21 12:39:43.449618 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:43.449515 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs\") pod \"network-metrics-daemon-z65kc\" (UID: \"eb0efdf9-1311-4257-9f40-3c35bee29d66\") " pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:43.449792 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:43.449681 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:43.449792 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:43.449752 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs podName:eb0efdf9-1311-4257-9f40-3c35bee29d66 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:47.449733603 +0000 UTC m=+10.068026551 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs") pod "network-metrics-daemon-z65kc" (UID: "eb0efdf9-1311-4257-9f40-3c35bee29d66") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:43.551309 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:43.550734 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret\") pod \"global-pull-secret-syncer-vtvm8\" (UID: \"a807b465-26c5-46ad-8abb-09a4f7716bee\") " pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:43.551309 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:43.550900 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:43.551309 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:43.550959 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret podName:a807b465-26c5-46ad-8abb-09a4f7716bee nodeName:}" failed. No retries permitted until 2026-07-21 12:39:47.550941025 +0000 UTC m=+10.169233956 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret") pod "global-pull-secret-syncer-vtvm8" (UID: "a807b465-26c5-46ad-8abb-09a4f7716bee") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:43.651831 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:43.651795 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lvlz\" (UniqueName: \"kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz\") pod \"network-check-target-jblzc\" (UID: \"2026fc18-7ca0-4d3e-8adf-aca525a12986\") " pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:39:43.652004 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:43.651981 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:39:43.652004 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:43.652002 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:39:43.652125 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:43.652014 2555 projected.go:196] Error preparing data for projected volume kube-api-access-7lvlz for pod openshift-network-diagnostics/network-check-target-jblzc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:39:43.652125 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:43.652072 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz podName:2026fc18-7ca0-4d3e-8adf-aca525a12986 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:47.652052182 +0000 UTC m=+10.270345123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7lvlz" (UniqueName: "kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz") pod "network-check-target-jblzc" (UID: "2026fc18-7ca0-4d3e-8adf-aca525a12986") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:39:43.922138 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:43.922042 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:43.922307 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:43.922225 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z65kc" podUID="eb0efdf9-1311-4257-9f40-3c35bee29d66" Jul 21 12:39:43.922648 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:43.922628 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:43.922748 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:43.922726 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtvm8" podUID="a807b465-26c5-46ad-8abb-09a4f7716bee" Jul 21 12:39:43.922824 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:43.922792 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:39:43.922879 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:43.922843 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jblzc" podUID="2026fc18-7ca0-4d3e-8adf-aca525a12986" Jul 21 12:39:45.919755 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:45.919588 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:39:45.919755 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:45.919677 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:45.919755 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:45.919695 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:45.920246 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:45.919842 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtvm8" podUID="a807b465-26c5-46ad-8abb-09a4f7716bee" Jul 21 12:39:45.920341 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:45.920308 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z65kc" podUID="eb0efdf9-1311-4257-9f40-3c35bee29d66" Jul 21 12:39:45.920458 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:45.920384 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jblzc" podUID="2026fc18-7ca0-4d3e-8adf-aca525a12986" Jul 21 12:39:47.488829 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:47.488788 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs\") pod \"network-metrics-daemon-z65kc\" (UID: \"eb0efdf9-1311-4257-9f40-3c35bee29d66\") " pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:47.489247 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:47.488971 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:47.489247 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:47.489051 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs podName:eb0efdf9-1311-4257-9f40-3c35bee29d66 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:55.489027324 +0000 UTC m=+18.107320299 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs") pod "network-metrics-daemon-z65kc" (UID: "eb0efdf9-1311-4257-9f40-3c35bee29d66") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:47.589345 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:47.589301 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret\") pod \"global-pull-secret-syncer-vtvm8\" (UID: \"a807b465-26c5-46ad-8abb-09a4f7716bee\") " pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:47.589524 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:47.589480 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:47.589586 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:47.589544 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret podName:a807b465-26c5-46ad-8abb-09a4f7716bee nodeName:}" failed. No retries permitted until 2026-07-21 12:39:55.589525518 +0000 UTC m=+18.207818447 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret") pod "global-pull-secret-syncer-vtvm8" (UID: "a807b465-26c5-46ad-8abb-09a4f7716bee") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:47.690757 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:47.690711 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lvlz\" (UniqueName: \"kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz\") pod \"network-check-target-jblzc\" (UID: \"2026fc18-7ca0-4d3e-8adf-aca525a12986\") " pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:39:47.690949 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:47.690877 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:39:47.690949 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:47.690898 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:39:47.690949 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:47.690913 2555 projected.go:196] Error preparing data for projected volume kube-api-access-7lvlz for pod openshift-network-diagnostics/network-check-target-jblzc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:39:47.691080 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:47.690973 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz podName:2026fc18-7ca0-4d3e-8adf-aca525a12986 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:55.690955423 +0000 UTC m=+18.309248351 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7lvlz" (UniqueName: "kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz") pod "network-check-target-jblzc" (UID: "2026fc18-7ca0-4d3e-8adf-aca525a12986") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:39:47.921265 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:47.921190 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:47.921397 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:47.921281 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtvm8" podUID="a807b465-26c5-46ad-8abb-09a4f7716bee" Jul 21 12:39:47.921578 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:47.921563 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:39:47.921660 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:47.921646 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jblzc" podUID="2026fc18-7ca0-4d3e-8adf-aca525a12986" Jul 21 12:39:47.921903 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:47.921779 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:47.921903 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:47.921873 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z65kc" podUID="eb0efdf9-1311-4257-9f40-3c35bee29d66" Jul 21 12:39:49.920273 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:49.920203 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:49.920273 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:49.920203 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:49.920841 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:49.920357 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z65kc" podUID="eb0efdf9-1311-4257-9f40-3c35bee29d66" Jul 21 12:39:49.920841 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:49.920224 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:39:49.920945 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:49.920849 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jblzc" podUID="2026fc18-7ca0-4d3e-8adf-aca525a12986" Jul 21 12:39:49.920945 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:49.920932 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtvm8" podUID="a807b465-26c5-46ad-8abb-09a4f7716bee" Jul 21 12:39:51.919931 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:51.919891 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:51.920359 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:51.919888 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:39:51.920359 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:51.920011 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z65kc" podUID="eb0efdf9-1311-4257-9f40-3c35bee29d66" Jul 21 12:39:51.920359 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:51.920032 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:51.920359 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:51.920130 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jblzc" podUID="2026fc18-7ca0-4d3e-8adf-aca525a12986" Jul 21 12:39:51.920359 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:51.920220 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtvm8" podUID="a807b465-26c5-46ad-8abb-09a4f7716bee" Jul 21 12:39:53.919660 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:53.919616 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:53.919660 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:53.919662 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:53.920188 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:53.919616 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:39:53.920188 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:53.919764 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z65kc" podUID="eb0efdf9-1311-4257-9f40-3c35bee29d66" Jul 21 12:39:53.920188 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:53.919808 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jblzc" podUID="2026fc18-7ca0-4d3e-8adf-aca525a12986" Jul 21 12:39:53.920188 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:53.919848 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtvm8" podUID="a807b465-26c5-46ad-8abb-09a4f7716bee" Jul 21 12:39:55.547457 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:55.547425 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs\") pod \"network-metrics-daemon-z65kc\" (UID: \"eb0efdf9-1311-4257-9f40-3c35bee29d66\") " pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:55.547932 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:55.547544 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:55.547932 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:55.547625 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs podName:eb0efdf9-1311-4257-9f40-3c35bee29d66 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:11.547607786 +0000 UTC m=+34.165900729 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs") pod "network-metrics-daemon-z65kc" (UID: "eb0efdf9-1311-4257-9f40-3c35bee29d66") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:39:55.648431 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:55.648391 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret\") pod \"global-pull-secret-syncer-vtvm8\" (UID: \"a807b465-26c5-46ad-8abb-09a4f7716bee\") " pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:55.648618 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:55.648532 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:55.648618 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:55.648615 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret podName:a807b465-26c5-46ad-8abb-09a4f7716bee nodeName:}" failed. No retries permitted until 2026-07-21 12:40:11.648581001 +0000 UTC m=+34.266873940 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret") pod "global-pull-secret-syncer-vtvm8" (UID: "a807b465-26c5-46ad-8abb-09a4f7716bee") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:39:55.749667 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:55.749633 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lvlz\" (UniqueName: \"kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz\") pod \"network-check-target-jblzc\" (UID: \"2026fc18-7ca0-4d3e-8adf-aca525a12986\") " pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:39:55.749860 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:55.749779 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:39:55.749860 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:55.749798 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:39:55.749860 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:55.749807 2555 projected.go:196] Error preparing data for projected volume kube-api-access-7lvlz for pod openshift-network-diagnostics/network-check-target-jblzc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:39:55.749995 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:55.749868 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz podName:2026fc18-7ca0-4d3e-8adf-aca525a12986 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:11.749848 +0000 UTC m=+34.368140926 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7lvlz" (UniqueName: "kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz") pod "network-check-target-jblzc" (UID: "2026fc18-7ca0-4d3e-8adf-aca525a12986") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:39:55.919802 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:55.919731 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:55.919802 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:55.919756 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:39:55.919950 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:55.919844 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z65kc" podUID="eb0efdf9-1311-4257-9f40-3c35bee29d66" Jul 21 12:39:55.919950 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:55.919880 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:55.920019 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:55.919956 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtvm8" podUID="a807b465-26c5-46ad-8abb-09a4f7716bee" Jul 21 12:39:55.920052 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:55.920010 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jblzc" podUID="2026fc18-7ca0-4d3e-8adf-aca525a12986" Jul 21 12:39:57.921193 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:57.920666 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:39:57.921193 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:57.920760 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:57.921193 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:57.920777 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jblzc" podUID="2026fc18-7ca0-4d3e-8adf-aca525a12986" Jul 21 12:39:57.921193 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:57.920881 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z65kc" podUID="eb0efdf9-1311-4257-9f40-3c35bee29d66" Jul 21 12:39:57.921193 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:57.920915 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:57.921193 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:57.920984 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtvm8" podUID="a807b465-26c5-46ad-8abb-09a4f7716bee" Jul 21 12:39:59.919718 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:59.919687 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:39:59.919718 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:59.919710 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:39:59.920354 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:59.919814 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jblzc" podUID="2026fc18-7ca0-4d3e-8adf-aca525a12986" Jul 21 12:39:59.920354 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:39:59.919896 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:39:59.920354 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:59.919899 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z65kc" podUID="eb0efdf9-1311-4257-9f40-3c35bee29d66" Jul 21 12:39:59.920354 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:39:59.919969 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtvm8" podUID="a807b465-26c5-46ad-8abb-09a4f7716bee" Jul 21 12:40:01.029459 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.029216 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" event={"ID":"fbe1e799-15a0-4706-a471-f2c8a2e61675","Type":"ContainerStarted","Data":"5ea868d3f07833709e71a4ddea790e821066966131825a2c2a0676da6a7fc953"} Jul 21 12:40:01.030833 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.030808 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x72xs" event={"ID":"aaa90a3d-1518-43bb-bff2-319c194a66ea","Type":"ContainerStarted","Data":"f4b758bca8330413b8c4d31bcfa33b6807222e8bfb285abed832e9b56c84f230"} Jul 21 12:40:01.032212 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.032186 2555 generic.go:356] "Generic (PLEG): container finished" podID="5348e68a-685f-42bb-bceb-2f325e562a66" containerID="11bb39b23af15a5629e737d48292760739d44b7c4d5a9eaff728a53af35e515f" exitCode=0 Jul 21 12:40:01.032305 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.032263 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-npbb8" event={"ID":"5348e68a-685f-42bb-bceb-2f325e562a66","Type":"ContainerDied","Data":"11bb39b23af15a5629e737d48292760739d44b7c4d5a9eaff728a53af35e515f"} Jul 21 12:40:01.034527 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.034511 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 12:40:01.034962 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.034939 2555 generic.go:356] "Generic (PLEG): container finished" podID="313167cc-d4ec-4ab0-9f12-b66389ad21fb" containerID="71d55a4d8406b097370e3e128f5b6823739fbbc8869cd66020e83492d33d8fb6" exitCode=1 Jul 21 12:40:01.035048 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.035005 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" event={"ID":"313167cc-d4ec-4ab0-9f12-b66389ad21fb","Type":"ContainerStarted","Data":"975e63541ac5db72371972ea34e2b9134640a60a63ec1d78105b08c62c1d22aa"} Jul 21 12:40:01.035101 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.035085 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" event={"ID":"313167cc-d4ec-4ab0-9f12-b66389ad21fb","Type":"ContainerStarted","Data":"5ceea3f88caa4edec60ff14f0da5742d9463dac0aecdf0d51c7c219be215d63e"} Jul 21 12:40:01.035149 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.035137 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" event={"ID":"313167cc-d4ec-4ab0-9f12-b66389ad21fb","Type":"ContainerDied","Data":"71d55a4d8406b097370e3e128f5b6823739fbbc8869cd66020e83492d33d8fb6"} Jul 21 12:40:01.035213 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.035197 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" event={"ID":"313167cc-d4ec-4ab0-9f12-b66389ad21fb","Type":"ContainerStarted","Data":"1e9d955b8b8227912fceb53f7568ae97c090b54096c500e4b143110aa85581ba"} Jul 21 12:40:01.036555 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.036535 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" event={"ID":"f54b837e-c8bb-4787-8667-4f860ffe6c5d","Type":"ContainerStarted","Data":"e23e57171bbc3b80e41a75ac1bc42704b1f8a92a84daf7b287ea7d17b8cc566e"} Jul 21 12:40:01.037784 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.037764 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-chngz" event={"ID":"58089888-e584-4b42-ae32-e8074321df87","Type":"ContainerStarted","Data":"6d7a656de2327c794bdbba70ebfd1c670776b7060d145b18fc74c14a99cbbcbd"} Jul 21 12:40:01.039215 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.039195 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dszxc" event={"ID":"2564902e-b620-4b49-9e45-43e1b7a43f87","Type":"ContainerStarted","Data":"48467cc311c2718e6dc1b94b98c862e8283a454370ebe99e4a755d23e9e46e96"} Jul 21 12:40:01.040382 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.040360 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v98xb" event={"ID":"ba7b78ab-59ae-485e-93cc-8005bd5b9a69","Type":"ContainerStarted","Data":"c8ff6c3f61e65c48d4f6a43a90e346726224582d911f258b36b3a43b740dfd3b"} Jul 21 12:40:01.051453 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.051402 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-39.ec2.internal" podStartSLOduration=22.051391472 podStartE2EDuration="22.051391472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:39:43.01118569 +0000 UTC m=+5.629478640" watchObservedRunningTime="2026-07-21 12:40:01.051391472 +0000 UTC m=+23.669684419" Jul 21 12:40:01.051796 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.051773 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kxdvv" podStartSLOduration=3.504294526 podStartE2EDuration="23.051766299s" totalImagesPullingTime="19.547471773s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:01.051584419 +0000 UTC m=+23.669877367" watchObservedRunningTime="2026-07-21 12:40:01.051766299 +0000 UTC m=+23.670059247" Jul 21 12:40:01.094505 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.094463 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-v98xb" podStartSLOduration=3.545775151 podStartE2EDuration="23.09444904s" totalImagesPullingTime="19.548673889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:01.094235571 +0000 UTC m=+23.712528518" watchObservedRunningTime="2026-07-21 12:40:01.09444904 +0000 UTC m=+23.712741988" Jul 21 12:40:01.109557 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.109517 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-chngz" podStartSLOduration=3.548803925 podStartE2EDuration="23.109502338s" totalImagesPullingTime="19.560698413s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:01.109177505 +0000 UTC m=+23.727470453" watchObservedRunningTime="2026-07-21 12:40:01.109502338 +0000 UTC m=+23.727795286" Jul 21 12:40:01.156340 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.156283 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dszxc" podStartSLOduration=4.560316521 podStartE2EDuration="24.156266909s" totalImagesPullingTime="19.595950388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:01.155855889 +0000 UTC m=+23.774148852" watchObservedRunningTime="2026-07-21 12:40:01.156266909 +0000 UTC m=+23.774559856" Jul 21 12:40:01.171253 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.171148 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-x72xs" podStartSLOduration=3.610527701 podStartE2EDuration="23.17113094s" totalImagesPullingTime="19.560603239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:39:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:01.170393771 +0000 UTC m=+23.788686720" watchObservedRunningTime="2026-07-21 12:40:01.17113094 +0000 UTC m=+23.789423887" Jul 21 12:40:01.920146 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.920097 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:40:01.920440 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.920101 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:40:01.920440 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:01.920262 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z65kc" podUID="eb0efdf9-1311-4257-9f40-3c35bee29d66" Jul 21 12:40:01.920440 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:01.920098 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:40:01.920440 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:01.920320 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtvm8" podUID="a807b465-26c5-46ad-8abb-09a4f7716bee" Jul 21 12:40:01.920440 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:01.920405 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jblzc" podUID="2026fc18-7ca0-4d3e-8adf-aca525a12986" Jul 21 12:40:02.045080 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:02.045041 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7q8jz" event={"ID":"2e3b06f6-c94b-485a-8d0f-bf23abb58edd","Type":"ContainerStarted","Data":"6480716bc0278c35929ff11c4fdc8823c9265bf72923d76371f22d90bcf87f7b"} Jul 21 12:40:02.048345 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:02.048318 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 12:40:02.049369 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:02.049341 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" event={"ID":"313167cc-d4ec-4ab0-9f12-b66389ad21fb","Type":"ContainerStarted","Data":"cc6414fcac17add3ebe76ad81c578f52956a4008d7bca610bdc52f256eaa69cb"} Jul 21 12:40:02.049473 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:02.049406 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" event={"ID":"313167cc-d4ec-4ab0-9f12-b66389ad21fb","Type":"ContainerStarted","Data":"78341446ec233d4ba62f216a81b2affd690562b0c342fc12b3584cd353b849bd"} Jul 21 12:40:02.538132 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:02.538096 2555 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 21 12:40:02.847911 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:02.847757 2555 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T12:40:02.538116012Z","UUID":"a97eb950-1325-4046-b20d-b33341916a4c","Handler":null,"Name":"","Endpoint":""} Jul 21 12:40:02.849477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:02.849457 2555 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 21 12:40:02.849694 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:02.849499 2555 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 21 12:40:03.053548 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:03.053506 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" event={"ID":"f54b837e-c8bb-4787-8667-4f860ffe6c5d","Type":"ContainerStarted","Data":"ed55a73d78a1df35bef5c4c8b0ef9c478e1bfc81d2194f268eec04d2d80b7d82"} Jul 21 12:40:03.920401 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:03.920367 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:40:03.920401 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:03.920403 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:40:03.920663 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:03.920369 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:40:03.920663 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:03.920488 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jblzc" podUID="2026fc18-7ca0-4d3e-8adf-aca525a12986" Jul 21 12:40:03.920768 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:03.920646 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z65kc" podUID="eb0efdf9-1311-4257-9f40-3c35bee29d66" Jul 21 12:40:03.920768 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:03.920747 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtvm8" podUID="a807b465-26c5-46ad-8abb-09a4f7716bee" Jul 21 12:40:04.031014 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:04.030978 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-x72xs" Jul 21 12:40:04.031545 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:04.031529 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-x72xs" Jul 21 12:40:04.054928 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:04.054874 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7q8jz" podStartSLOduration=7.508048519 podStartE2EDuration="27.054851183s" totalImagesPullingTime="19.546802664s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:02.063121087 +0000 UTC m=+24.681414035" watchObservedRunningTime="2026-07-21 12:40:04.054851183 +0000 UTC m=+26.673144132" Jul 21 12:40:04.063672 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:04.063646 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 12:40:04.064176 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:04.064135 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" event={"ID":"313167cc-d4ec-4ab0-9f12-b66389ad21fb","Type":"ContainerStarted","Data":"f71261ef84a340ce7d25062ec6d3eecfd9adb121d8d4d14d82ed587f1d94a033"} Jul 21 12:40:04.064296 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:04.064284 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-x72xs" Jul 21 12:40:04.064771 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:04.064752 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x72xs" Jul 21 12:40:05.920198 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:05.919835 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:40:05.921190 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:05.920301 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z65kc" podUID="eb0efdf9-1311-4257-9f40-3c35bee29d66" Jul 21 12:40:05.921190 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:05.919974 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:40:05.921190 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:05.920399 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtvm8" podUID="a807b465-26c5-46ad-8abb-09a4f7716bee" Jul 21 12:40:05.921190 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:05.919835 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:40:05.921190 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:05.920470 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jblzc" podUID="2026fc18-7ca0-4d3e-8adf-aca525a12986" Jul 21 12:40:06.071305 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:06.071269 2555 generic.go:356] "Generic (PLEG): container finished" podID="5348e68a-685f-42bb-bceb-2f325e562a66" containerID="a73c77148a63237b9011aaeb4bc66195edab42268e31ed185e67f8d9e2b6c84e" exitCode=0 Jul 21 12:40:06.071477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:06.071373 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-npbb8" event={"ID":"5348e68a-685f-42bb-bceb-2f325e562a66","Type":"ContainerDied","Data":"a73c77148a63237b9011aaeb4bc66195edab42268e31ed185e67f8d9e2b6c84e"} Jul 21 12:40:06.075893 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:06.075876 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 12:40:06.076333 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:06.076309 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" event={"ID":"313167cc-d4ec-4ab0-9f12-b66389ad21fb","Type":"ContainerStarted","Data":"10c0d33df858772e4c1e33f2f428272f17fb554c1f39743279230f17d55e65d6"} Jul 21 12:40:06.076724 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:06.076706 2555 scope.go:117] "RemoveContainer" containerID="71d55a4d8406b097370e3e128f5b6823739fbbc8869cd66020e83492d33d8fb6" Jul 21 12:40:07.082436 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:07.082409 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 12:40:07.082979 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:07.082817 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" event={"ID":"313167cc-d4ec-4ab0-9f12-b66389ad21fb","Type":"ContainerStarted","Data":"57a38b0893823cf9a13ac8288df11f1b6c12d4d9cb9c349665b577fbf4dd44b1"} Jul 21 12:40:07.083250 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:07.083222 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:40:07.083350 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:07.083314 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:40:07.083431 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:07.083354 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:40:07.085498 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:07.085462 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" event={"ID":"f54b837e-c8bb-4787-8667-4f860ffe6c5d","Type":"ContainerStarted","Data":"2dedac9a9c44d9aa70635e6a48882ce1ddbecc2502826f88eb50d5767e4bedfd"} Jul 21 12:40:07.099801 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:07.099761 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:40:07.099986 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:07.099933 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:40:07.115908 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:07.115848 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" podStartSLOduration=9.495008408 podStartE2EDuration="29.115828389s" totalImagesPullingTime="19.620819981s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:07.114961613 +0000 UTC m=+29.733254574" watchObservedRunningTime="2026-07-21 12:40:07.115828389 +0000 UTC m=+29.734121338" Jul 21 12:40:07.626058 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:07.626009 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rb99c" podStartSLOduration=4.383604237 podStartE2EDuration="29.625991751s" totalImagesPullingTime="25.242387514s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:07.164134155 +0000 UTC m=+29.782427101" watchObservedRunningTime="2026-07-21 12:40:07.625991751 +0000 UTC m=+30.244284698" Jul 21 12:40:07.626305 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:07.626286 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vtvm8"] Jul 21 12:40:07.626425 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:07.626393 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:40:07.626497 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:07.626481 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtvm8" podUID="a807b465-26c5-46ad-8abb-09a4f7716bee" Jul 21 12:40:07.629438 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:07.629405 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jblzc"] Jul 21 12:40:07.629571 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:07.629513 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:40:07.629636 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:07.629613 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jblzc" podUID="2026fc18-7ca0-4d3e-8adf-aca525a12986" Jul 21 12:40:07.632421 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:07.632386 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-z65kc"] Jul 21 12:40:07.632548 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:07.632507 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:40:07.632645 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:07.632627 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z65kc" podUID="eb0efdf9-1311-4257-9f40-3c35bee29d66" Jul 21 12:40:08.090505 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:08.090469 2555 generic.go:356] "Generic (PLEG): container finished" podID="5348e68a-685f-42bb-bceb-2f325e562a66" containerID="e369c065f80779fa6670571a4978892f8195b0e1dc054e2aeb037be5738fdd63" exitCode=0 Jul 21 12:40:08.091089 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:08.090554 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-npbb8" event={"ID":"5348e68a-685f-42bb-bceb-2f325e562a66","Type":"ContainerDied","Data":"e369c065f80779fa6670571a4978892f8195b0e1dc054e2aeb037be5738fdd63"} Jul 21 12:40:08.919558 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:08.919529 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:40:08.919734 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:08.919531 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:40:08.919734 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:08.919646 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jblzc" podUID="2026fc18-7ca0-4d3e-8adf-aca525a12986" Jul 21 12:40:08.919734 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:08.919531 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:40:08.919734 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:08.919719 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z65kc" podUID="eb0efdf9-1311-4257-9f40-3c35bee29d66" Jul 21 12:40:08.919869 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:08.919778 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtvm8" podUID="a807b465-26c5-46ad-8abb-09a4f7716bee" Jul 21 12:40:10.097527 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:10.097328 2555 generic.go:356] "Generic (PLEG): container finished" podID="5348e68a-685f-42bb-bceb-2f325e562a66" containerID="37b54b3ec0754c6aca6942e4d30f90270badc122e24edc29df738182410bfc50" exitCode=0 Jul 21 12:40:10.098015 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:10.097415 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-npbb8" event={"ID":"5348e68a-685f-42bb-bceb-2f325e562a66","Type":"ContainerDied","Data":"37b54b3ec0754c6aca6942e4d30f90270badc122e24edc29df738182410bfc50"} Jul 21 12:40:10.919809 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:10.919774 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:40:10.920002 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:10.919774 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:40:10.920002 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:10.919914 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z65kc" podUID="eb0efdf9-1311-4257-9f40-3c35bee29d66" Jul 21 12:40:10.920002 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:10.919774 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:40:10.920168 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:10.919992 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtvm8" podUID="a807b465-26c5-46ad-8abb-09a4f7716bee" Jul 21 12:40:10.920168 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:10.920113 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jblzc" podUID="2026fc18-7ca0-4d3e-8adf-aca525a12986" Jul 21 12:40:11.566630 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:11.566494 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs\") pod \"network-metrics-daemon-z65kc\" (UID: \"eb0efdf9-1311-4257-9f40-3c35bee29d66\") " pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:40:11.567217 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:11.566658 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:40:11.567217 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:11.566734 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs podName:eb0efdf9-1311-4257-9f40-3c35bee29d66 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:43.566713584 +0000 UTC m=+66.185006514 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs") pod "network-metrics-daemon-z65kc" (UID: "eb0efdf9-1311-4257-9f40-3c35bee29d66") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 12:40:11.667141 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:11.667100 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret\") pod \"global-pull-secret-syncer-vtvm8\" (UID: \"a807b465-26c5-46ad-8abb-09a4f7716bee\") " pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:40:11.667342 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:11.667244 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 12:40:11.667342 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:11.667314 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret podName:a807b465-26c5-46ad-8abb-09a4f7716bee nodeName:}" failed. No retries permitted until 2026-07-21 12:40:43.667294255 +0000 UTC m=+66.285587203 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret") pod "global-pull-secret-syncer-vtvm8" (UID: "a807b465-26c5-46ad-8abb-09a4f7716bee") : object "kube-system"/"original-pull-secret" not registered Jul 21 12:40:11.767902 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:11.767870 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lvlz\" (UniqueName: \"kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz\") pod \"network-check-target-jblzc\" (UID: \"2026fc18-7ca0-4d3e-8adf-aca525a12986\") " pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:40:11.768070 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:11.768031 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 12:40:11.768070 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:11.768048 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 12:40:11.768070 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:11.768060 2555 projected.go:196] Error preparing data for projected volume kube-api-access-7lvlz for pod openshift-network-diagnostics/network-check-target-jblzc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:40:11.768207 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:11.768127 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz podName:2026fc18-7ca0-4d3e-8adf-aca525a12986 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:43.768107445 +0000 UTC m=+66.386400372 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7lvlz" (UniqueName: "kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz") pod "network-check-target-jblzc" (UID: "2026fc18-7ca0-4d3e-8adf-aca525a12986") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 12:40:12.919532 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:12.919495 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:40:12.920104 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:12.919636 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:40:12.920104 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:12.919756 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jblzc" podUID="2026fc18-7ca0-4d3e-8adf-aca525a12986" Jul 21 12:40:12.920104 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:12.919647 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:40:12.920104 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:12.919644 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vtvm8" podUID="a807b465-26c5-46ad-8abb-09a4f7716bee" Jul 21 12:40:12.920104 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:12.919897 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z65kc" podUID="eb0efdf9-1311-4257-9f40-3c35bee29d66" Jul 21 12:40:13.175098 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.175024 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-39.ec2.internal" event="NodeReady" Jul 21 12:40:13.175267 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.175175 2555 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 21 12:40:13.215418 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.215378 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7fd754cdd7-bgzfh"] Jul 21 12:40:13.218213 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.218191 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.220881 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.220857 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 21 12:40:13.220881 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.220870 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 21 12:40:13.221051 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.220914 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 21 12:40:13.221180 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.221163 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-v4mnz\"" Jul 21 12:40:13.226365 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.226345 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 21 12:40:13.231012 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.230988 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gcm9k"] Jul 21 12:40:13.233166 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.233143 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:13.235157 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.235133 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 21 12:40:13.235364 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.235343 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fd754cdd7-bgzfh"] Jul 21 12:40:13.235440 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.235362 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 21 12:40:13.235440 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.235386 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-f6p9g"] Jul 21 12:40:13.235801 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.235783 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h5b4t\"" Jul 21 12:40:13.237553 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.237534 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f6p9g" Jul 21 12:40:13.239511 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.239483 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 21 12:40:13.239840 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.239664 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7n7fg\"" Jul 21 12:40:13.239840 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.239730 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 21 12:40:13.240007 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.239995 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 21 12:40:13.247829 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.247796 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gcm9k"] Jul 21 12:40:13.250066 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.250042 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f6p9g"] Jul 21 12:40:13.382337 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.382294 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6e594b6-715f-4157-b411-94fbf7a3bdce-image-registry-private-configuration\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.382530 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.382346 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns445\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-kube-api-access-ns445\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.382530 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.382371 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls\") pod \"dns-default-gcm9k\" (UID: \"de273e85-f101-43f9-96ec-b7102d29ac49\") " pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:13.382530 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.382401 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6e594b6-715f-4157-b411-94fbf7a3bdce-ca-trust-extracted\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.382530 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.382443 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pltq9\" (UniqueName: \"kubernetes.io/projected/de273e85-f101-43f9-96ec-b7102d29ac49-kube-api-access-pltq9\") pod \"dns-default-gcm9k\" (UID: \"de273e85-f101-43f9-96ec-b7102d29ac49\") " pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:13.382530 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.382508 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnw5b\" (UniqueName: \"kubernetes.io/projected/02d7c116-6592-44ac-89f4-8cba0c30c18b-kube-api-access-wnw5b\") pod \"ingress-canary-f6p9g\" (UID: \"02d7c116-6592-44ac-89f4-8cba0c30c18b\") " pod="openshift-ingress-canary/ingress-canary-f6p9g" Jul 21 12:40:13.382823 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.382538 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6e594b6-715f-4157-b411-94fbf7a3bdce-installation-pull-secrets\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.382823 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.382618 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.382823 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.382676 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/de273e85-f101-43f9-96ec-b7102d29ac49-config-volume\") pod \"dns-default-gcm9k\" (UID: \"de273e85-f101-43f9-96ec-b7102d29ac49\") " pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:13.382823 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.382707 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-certificates\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.382823 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.382728 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6e594b6-715f-4157-b411-94fbf7a3bdce-trusted-ca\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.382823 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.382753 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-bound-sa-token\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.382823 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.382771 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de273e85-f101-43f9-96ec-b7102d29ac49-tmp-dir\") pod \"dns-default-gcm9k\" (UID: \"de273e85-f101-43f9-96ec-b7102d29ac49\") " pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:13.382823 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.382804 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/02d7c116-6592-44ac-89f4-8cba0c30c18b-cert\") pod \"ingress-canary-f6p9g\" (UID: \"02d7c116-6592-44ac-89f4-8cba0c30c18b\") " pod="openshift-ingress-canary/ingress-canary-f6p9g" Jul 21 12:40:13.483736 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.483642 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.483736 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.483709 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/de273e85-f101-43f9-96ec-b7102d29ac49-config-volume\") pod \"dns-default-gcm9k\" (UID: \"de273e85-f101-43f9-96ec-b7102d29ac49\") " pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:13.483974 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.483747 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-certificates\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.483974 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.483774 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6e594b6-715f-4157-b411-94fbf7a3bdce-trusted-ca\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.483974 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:13.483818 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 12:40:13.483974 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:13.483841 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fd754cdd7-bgzfh: secret "image-registry-tls" not found Jul 21 12:40:13.483974 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.483824 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-bound-sa-token\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.483974 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:13.483904 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls podName:b6e594b6-715f-4157-b411-94fbf7a3bdce nodeName:}" failed. No retries permitted until 2026-07-21 12:40:13.983884359 +0000 UTC m=+36.602177306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls") pod "image-registry-7fd754cdd7-bgzfh" (UID: "b6e594b6-715f-4157-b411-94fbf7a3bdce") : secret "image-registry-tls" not found Jul 21 12:40:13.483974 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.483968 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de273e85-f101-43f9-96ec-b7102d29ac49-tmp-dir\") pod \"dns-default-gcm9k\" (UID: \"de273e85-f101-43f9-96ec-b7102d29ac49\") " pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:13.484322 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.484013 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/02d7c116-6592-44ac-89f4-8cba0c30c18b-cert\") pod \"ingress-canary-f6p9g\" (UID: \"02d7c116-6592-44ac-89f4-8cba0c30c18b\") " pod="openshift-ingress-canary/ingress-canary-f6p9g" Jul 21 12:40:13.484322 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.484073 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6e594b6-715f-4157-b411-94fbf7a3bdce-image-registry-private-configuration\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.484322 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.484100 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns445\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-kube-api-access-ns445\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.484322 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.484126 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls\") pod \"dns-default-gcm9k\" (UID: \"de273e85-f101-43f9-96ec-b7102d29ac49\") " pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:13.484322 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.484166 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6e594b6-715f-4157-b411-94fbf7a3bdce-ca-trust-extracted\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.484322 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.484212 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pltq9\" (UniqueName: \"kubernetes.io/projected/de273e85-f101-43f9-96ec-b7102d29ac49-kube-api-access-pltq9\") pod \"dns-default-gcm9k\" (UID: \"de273e85-f101-43f9-96ec-b7102d29ac49\") " pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:13.484322 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.484248 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnw5b\" (UniqueName: \"kubernetes.io/projected/02d7c116-6592-44ac-89f4-8cba0c30c18b-kube-api-access-wnw5b\") pod \"ingress-canary-f6p9g\" (UID: \"02d7c116-6592-44ac-89f4-8cba0c30c18b\") " pod="openshift-ingress-canary/ingress-canary-f6p9g" Jul 21 12:40:13.484322 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.484272 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6e594b6-715f-4157-b411-94fbf7a3bdce-installation-pull-secrets\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.484322 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.484274 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de273e85-f101-43f9-96ec-b7102d29ac49-tmp-dir\") pod \"dns-default-gcm9k\" (UID: \"de273e85-f101-43f9-96ec-b7102d29ac49\") " pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:13.484750 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.484442 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-certificates\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.484750 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.484467 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/de273e85-f101-43f9-96ec-b7102d29ac49-config-volume\") pod \"dns-default-gcm9k\" (UID: \"de273e85-f101-43f9-96ec-b7102d29ac49\") " pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:13.484750 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.484668 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6e594b6-715f-4157-b411-94fbf7a3bdce-trusted-ca\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.484750 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.484694 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6e594b6-715f-4157-b411-94fbf7a3bdce-ca-trust-extracted\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.484943 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:13.484801 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:40:13.484943 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:13.484869 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls podName:de273e85-f101-43f9-96ec-b7102d29ac49 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:13.984851774 +0000 UTC m=+36.603144700 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls") pod "dns-default-gcm9k" (UID: "de273e85-f101-43f9-96ec-b7102d29ac49") : secret "dns-default-metrics-tls" not found Jul 21 12:40:13.489351 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.489328 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6e594b6-715f-4157-b411-94fbf7a3bdce-image-registry-private-configuration\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.489976 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.489954 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6e594b6-715f-4157-b411-94fbf7a3bdce-installation-pull-secrets\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.490074 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.490009 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/02d7c116-6592-44ac-89f4-8cba0c30c18b-cert\") pod \"ingress-canary-f6p9g\" (UID: \"02d7c116-6592-44ac-89f4-8cba0c30c18b\") " pod="openshift-ingress-canary/ingress-canary-f6p9g" Jul 21 12:40:13.495089 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.495069 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pltq9\" (UniqueName: \"kubernetes.io/projected/de273e85-f101-43f9-96ec-b7102d29ac49-kube-api-access-pltq9\") pod \"dns-default-gcm9k\" (UID: \"de273e85-f101-43f9-96ec-b7102d29ac49\") " pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:13.495518 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.495494 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-bound-sa-token\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.495855 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.495837 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns445\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-kube-api-access-ns445\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.495997 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.495980 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnw5b\" (UniqueName: \"kubernetes.io/projected/02d7c116-6592-44ac-89f4-8cba0c30c18b-kube-api-access-wnw5b\") pod \"ingress-canary-f6p9g\" (UID: \"02d7c116-6592-44ac-89f4-8cba0c30c18b\") " pod="openshift-ingress-canary/ingress-canary-f6p9g" Jul 21 12:40:13.555669 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.555633 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f6p9g" Jul 21 12:40:13.989488 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.989435 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls\") pod \"dns-default-gcm9k\" (UID: \"de273e85-f101-43f9-96ec-b7102d29ac49\") " pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:13.990138 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:13.989544 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:13.990138 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:13.989654 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:40:13.990138 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:13.989687 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 12:40:13.990138 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:13.989704 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fd754cdd7-bgzfh: secret "image-registry-tls" not found Jul 21 12:40:13.990138 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:13.989729 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls podName:de273e85-f101-43f9-96ec-b7102d29ac49 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:14.989711368 +0000 UTC m=+37.608004295 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls") pod "dns-default-gcm9k" (UID: "de273e85-f101-43f9-96ec-b7102d29ac49") : secret "dns-default-metrics-tls" not found Jul 21 12:40:13.990138 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:13.989749 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls podName:b6e594b6-715f-4157-b411-94fbf7a3bdce nodeName:}" failed. No retries permitted until 2026-07-21 12:40:14.989737305 +0000 UTC m=+37.608030231 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls") pod "image-registry-7fd754cdd7-bgzfh" (UID: "b6e594b6-715f-4157-b411-94fbf7a3bdce") : secret "image-registry-tls" not found Jul 21 12:40:14.475190 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:14.474935 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f6p9g"] Jul 21 12:40:14.479687 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:40:14.479662 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02d7c116_6592_44ac_89f4_8cba0c30c18b.slice/crio-4c0864abd96b10e9dde9e705792abf9f16d60b7e37e3cd82a0496716cc565f84 WatchSource:0}: Error finding container 4c0864abd96b10e9dde9e705792abf9f16d60b7e37e3cd82a0496716cc565f84: Status 404 returned error can't find the container with id 4c0864abd96b10e9dde9e705792abf9f16d60b7e37e3cd82a0496716cc565f84 Jul 21 12:40:14.920320 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:14.920286 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:40:14.920492 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:14.920286 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:40:14.920492 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:14.920286 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:40:14.923469 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:14.923443 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 12:40:14.923638 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:14.923482 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tqbcg\"" Jul 21 12:40:14.923638 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:14.923480 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rhfvk\"" Jul 21 12:40:14.923638 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:14.923447 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 12:40:14.923638 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:14.923485 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 12:40:14.923638 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:14.923519 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 12:40:14.999083 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:14.999043 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls\") pod \"dns-default-gcm9k\" (UID: \"de273e85-f101-43f9-96ec-b7102d29ac49\") " pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:14.999503 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:14.999121 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:14.999503 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:14.999211 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:40:14.999503 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:14.999265 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 12:40:14.999503 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:14.999280 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fd754cdd7-bgzfh: secret "image-registry-tls" not found Jul 21 12:40:14.999503 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:14.999287 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls podName:de273e85-f101-43f9-96ec-b7102d29ac49 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:16.999271179 +0000 UTC m=+39.617564107 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls") pod "dns-default-gcm9k" (UID: "de273e85-f101-43f9-96ec-b7102d29ac49") : secret "dns-default-metrics-tls" not found Jul 21 12:40:14.999503 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:14.999337 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls podName:b6e594b6-715f-4157-b411-94fbf7a3bdce nodeName:}" failed. No retries permitted until 2026-07-21 12:40:16.999321169 +0000 UTC m=+39.617614095 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls") pod "image-registry-7fd754cdd7-bgzfh" (UID: "b6e594b6-715f-4157-b411-94fbf7a3bdce") : secret "image-registry-tls" not found Jul 21 12:40:15.111512 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:15.111476 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f6p9g" event={"ID":"02d7c116-6592-44ac-89f4-8cba0c30c18b","Type":"ContainerStarted","Data":"4c0864abd96b10e9dde9e705792abf9f16d60b7e37e3cd82a0496716cc565f84"} Jul 21 12:40:15.114626 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:15.114583 2555 generic.go:356] "Generic (PLEG): container finished" podID="5348e68a-685f-42bb-bceb-2f325e562a66" containerID="258e7fb348d331b3547138045d173864728b911d04b0b38cae9ca9bc04b4801a" exitCode=0 Jul 21 12:40:15.114759 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:15.114628 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-npbb8" event={"ID":"5348e68a-685f-42bb-bceb-2f325e562a66","Type":"ContainerDied","Data":"258e7fb348d331b3547138045d173864728b911d04b0b38cae9ca9bc04b4801a"} Jul 21 12:40:16.120236 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:16.120195 2555 generic.go:356] "Generic (PLEG): container finished" podID="5348e68a-685f-42bb-bceb-2f325e562a66" containerID="91c002b8a8e5b382917e8f266627d7cdbd7f95cb10031dd7ee78676ebcb002bb" exitCode=0 Jul 21 12:40:16.120645 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:16.120276 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-npbb8" event={"ID":"5348e68a-685f-42bb-bceb-2f325e562a66","Type":"ContainerDied","Data":"91c002b8a8e5b382917e8f266627d7cdbd7f95cb10031dd7ee78676ebcb002bb"} Jul 21 12:40:17.015641 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:17.015381 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:17.015861 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:17.015712 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls\") pod \"dns-default-gcm9k\" (UID: \"de273e85-f101-43f9-96ec-b7102d29ac49\") " pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:17.015920 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:17.015566 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 12:40:17.015920 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:17.015882 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fd754cdd7-bgzfh: secret "image-registry-tls" not found Jul 21 12:40:17.016022 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:17.015934 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls podName:b6e594b6-715f-4157-b411-94fbf7a3bdce nodeName:}" failed. No retries permitted until 2026-07-21 12:40:21.01592059 +0000 UTC m=+43.634213516 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls") pod "image-registry-7fd754cdd7-bgzfh" (UID: "b6e594b6-715f-4157-b411-94fbf7a3bdce") : secret "image-registry-tls" not found Jul 21 12:40:17.016266 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:17.015862 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:40:17.016326 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:17.016298 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls podName:de273e85-f101-43f9-96ec-b7102d29ac49 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:21.016285154 +0000 UTC m=+43.634578081 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls") pod "dns-default-gcm9k" (UID: "de273e85-f101-43f9-96ec-b7102d29ac49") : secret "dns-default-metrics-tls" not found Jul 21 12:40:17.127968 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:17.127925 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-npbb8" event={"ID":"5348e68a-685f-42bb-bceb-2f325e562a66","Type":"ContainerStarted","Data":"2f2136fcee089e06a569f913833292fe423b20075cd938dc37a6f8e4528e0f45"} Jul 21 12:40:17.156526 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:17.156470 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-npbb8" podStartSLOduration=7.950638662 podStartE2EDuration="39.156446673s" totalImagesPullingTime="31.205808011s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:17.156106543 +0000 UTC m=+39.774399515" watchObservedRunningTime="2026-07-21 12:40:17.156446673 +0000 UTC m=+39.774739623" Jul 21 12:40:18.131215 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:18.131186 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f6p9g" event={"ID":"02d7c116-6592-44ac-89f4-8cba0c30c18b","Type":"ContainerStarted","Data":"790cca1c77cccc22158d9dee7294e81c4ad49757a9c57ee6927f659c994b9b03"} Jul 21 12:40:18.149786 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:18.149740 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-f6p9g" podStartSLOduration=2.032333735 podStartE2EDuration="5.149725966s" totalImagesPullingTime="3.117392231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:18.149112488 +0000 UTC m=+40.767405637" watchObservedRunningTime="2026-07-21 12:40:18.149725966 +0000 UTC m=+40.768018913" Jul 21 12:40:21.049387 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:21.049345 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls\") pod \"dns-default-gcm9k\" (UID: \"de273e85-f101-43f9-96ec-b7102d29ac49\") " pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:21.049797 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:21.049439 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:21.049797 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:21.049483 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:40:21.049797 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:21.049548 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls podName:de273e85-f101-43f9-96ec-b7102d29ac49 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:29.049531442 +0000 UTC m=+51.667824368 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls") pod "dns-default-gcm9k" (UID: "de273e85-f101-43f9-96ec-b7102d29ac49") : secret "dns-default-metrics-tls" not found Jul 21 12:40:21.049797 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:21.049548 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 12:40:21.049797 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:21.049562 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fd754cdd7-bgzfh: secret "image-registry-tls" not found Jul 21 12:40:21.049797 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:21.049624 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls podName:b6e594b6-715f-4157-b411-94fbf7a3bdce nodeName:}" failed. No retries permitted until 2026-07-21 12:40:29.049587699 +0000 UTC m=+51.667880624 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls") pod "image-registry-7fd754cdd7-bgzfh" (UID: "b6e594b6-715f-4157-b411-94fbf7a3bdce") : secret "image-registry-tls" not found Jul 21 12:40:22.664068 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.664035 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-qnc6t"] Jul 21 12:40:22.698298 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.698267 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-qnc6t"] Jul 21 12:40:22.698455 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.698361 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:40:22.700679 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.700652 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 21 12:40:22.700679 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.700657 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 21 12:40:22.701289 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.701270 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 21 12:40:22.701289 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.701279 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-tgstg\"" Jul 21 12:40:22.701619 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.701583 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 21 12:40:22.705999 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.705939 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 21 12:40:22.766811 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.766778 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-kprgx"] Jul 21 12:40:22.788275 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.788237 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6974856856-hqzjg"] Jul 21 12:40:22.788459 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.788375 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kprgx" Jul 21 12:40:22.790513 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.790309 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-h5wtg\"" Jul 21 12:40:22.814331 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.814292 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-kprgx"] Jul 21 12:40:22.814331 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.814341 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6974856856-hqzjg"] Jul 21 12:40:22.814574 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.814441 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:22.816281 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.816261 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 21 12:40:22.816438 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.816398 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-drcb2\"" Jul 21 12:40:22.816568 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.816548 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 21 12:40:22.816738 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.816547 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 21 12:40:22.816738 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.816616 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 21 12:40:22.816839 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.816826 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 21 12:40:22.816958 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.816947 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 21 12:40:22.866863 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.866828 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpx4k\" (UniqueName: \"kubernetes.io/projected/af138219-695f-4f5d-8a2f-df2c0a89f117-kube-api-access-dpx4k\") pod \"console-operator-565b7b569d-qnc6t\" (UID: \"af138219-695f-4f5d-8a2f-df2c0a89f117\") " pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:40:22.866863 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.866875 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af138219-695f-4f5d-8a2f-df2c0a89f117-serving-cert\") pod \"console-operator-565b7b569d-qnc6t\" (UID: \"af138219-695f-4f5d-8a2f-df2c0a89f117\") " pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:40:22.867071 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.866941 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af138219-695f-4f5d-8a2f-df2c0a89f117-trusted-ca\") pod \"console-operator-565b7b569d-qnc6t\" (UID: \"af138219-695f-4f5d-8a2f-df2c0a89f117\") " pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:40:22.867071 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.867010 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af138219-695f-4f5d-8a2f-df2c0a89f117-config\") pod \"console-operator-565b7b569d-qnc6t\" (UID: \"af138219-695f-4f5d-8a2f-df2c0a89f117\") " pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:40:22.968326 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.968236 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af138219-695f-4f5d-8a2f-df2c0a89f117-trusted-ca\") pod \"console-operator-565b7b569d-qnc6t\" (UID: \"af138219-695f-4f5d-8a2f-df2c0a89f117\") " pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:40:22.968326 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.968288 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af138219-695f-4f5d-8a2f-df2c0a89f117-config\") pod \"console-operator-565b7b569d-qnc6t\" (UID: \"af138219-695f-4f5d-8a2f-df2c0a89f117\") " pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:40:22.968545 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.968484 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-default-certificate\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:22.968545 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.968525 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-stats-auth\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:22.968662 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.968618 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:22.968743 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.968725 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpx4k\" (UniqueName: \"kubernetes.io/projected/af138219-695f-4f5d-8a2f-df2c0a89f117-kube-api-access-dpx4k\") pod \"console-operator-565b7b569d-qnc6t\" (UID: \"af138219-695f-4f5d-8a2f-df2c0a89f117\") " pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:40:22.968780 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.968759 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af138219-695f-4f5d-8a2f-df2c0a89f117-serving-cert\") pod \"console-operator-565b7b569d-qnc6t\" (UID: \"af138219-695f-4f5d-8a2f-df2c0a89f117\") " pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:40:22.968832 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.968782 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m294\" (UniqueName: \"kubernetes.io/projected/71802871-c712-4646-9b20-f6f78515bbc4-kube-api-access-6m294\") pod \"network-check-source-7bc5b6bf85-kprgx\" (UID: \"71802871-c712-4646-9b20-f6f78515bbc4\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kprgx" Jul 21 12:40:22.968868 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.968841 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:22.968919 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.968878 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr4f9\" (UniqueName: \"kubernetes.io/projected/9bb8c882-1f66-4054-8f88-80a3fd392806-kube-api-access-jr4f9\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:22.969170 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.969146 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af138219-695f-4f5d-8a2f-df2c0a89f117-config\") pod \"console-operator-565b7b569d-qnc6t\" (UID: \"af138219-695f-4f5d-8a2f-df2c0a89f117\") " pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:40:22.969337 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.969320 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af138219-695f-4f5d-8a2f-df2c0a89f117-trusted-ca\") pod \"console-operator-565b7b569d-qnc6t\" (UID: \"af138219-695f-4f5d-8a2f-df2c0a89f117\") " pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:40:22.972830 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.972803 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af138219-695f-4f5d-8a2f-df2c0a89f117-serving-cert\") pod \"console-operator-565b7b569d-qnc6t\" (UID: \"af138219-695f-4f5d-8a2f-df2c0a89f117\") " pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:40:22.977506 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:22.977485 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpx4k\" (UniqueName: \"kubernetes.io/projected/af138219-695f-4f5d-8a2f-df2c0a89f117-kube-api-access-dpx4k\") pod \"console-operator-565b7b569d-qnc6t\" (UID: \"af138219-695f-4f5d-8a2f-df2c0a89f117\") " pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:40:23.008340 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:23.008297 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:40:23.069738 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:23.069702 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr4f9\" (UniqueName: \"kubernetes.io/projected/9bb8c882-1f66-4054-8f88-80a3fd392806-kube-api-access-jr4f9\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:23.069925 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:23.069758 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-default-certificate\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:23.069925 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:23.069783 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-stats-auth\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:23.069925 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:23.069812 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:23.069925 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:23.069882 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m294\" (UniqueName: \"kubernetes.io/projected/71802871-c712-4646-9b20-f6f78515bbc4-kube-api-access-6m294\") pod \"network-check-source-7bc5b6bf85-kprgx\" (UID: \"71802871-c712-4646-9b20-f6f78515bbc4\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kprgx" Jul 21 12:40:23.070115 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:23.069980 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 12:40:23.070115 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:23.070028 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:23.070115 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:23.070052 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs podName:9bb8c882-1f66-4054-8f88-80a3fd392806 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:23.57003632 +0000 UTC m=+46.188329246 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs") pod "router-default-6974856856-hqzjg" (UID: "9bb8c882-1f66-4054-8f88-80a3fd392806") : secret "router-metrics-certs-default" not found Jul 21 12:40:23.070270 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:23.070155 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle podName:9bb8c882-1f66-4054-8f88-80a3fd392806 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:23.570135441 +0000 UTC m=+46.188428381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle") pod "router-default-6974856856-hqzjg" (UID: "9bb8c882-1f66-4054-8f88-80a3fd392806") : configmap references non-existent config key: service-ca.crt Jul 21 12:40:23.072717 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:23.072525 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-default-certificate\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:23.072852 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:23.072809 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-stats-auth\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:23.081280 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:23.081233 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr4f9\" (UniqueName: \"kubernetes.io/projected/9bb8c882-1f66-4054-8f88-80a3fd392806-kube-api-access-jr4f9\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:23.085703 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:23.085655 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m294\" (UniqueName: \"kubernetes.io/projected/71802871-c712-4646-9b20-f6f78515bbc4-kube-api-access-6m294\") pod \"network-check-source-7bc5b6bf85-kprgx\" (UID: \"71802871-c712-4646-9b20-f6f78515bbc4\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kprgx" Jul 21 12:40:23.097420 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:23.097374 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kprgx" Jul 21 12:40:23.157633 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:23.157586 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-qnc6t"] Jul 21 12:40:23.161198 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:40:23.161175 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf138219_695f_4f5d_8a2f_df2c0a89f117.slice/crio-db24675066306fa678e0d7584035d129bec4fde87accbabd2cea3bb784dfbbad WatchSource:0}: Error finding container db24675066306fa678e0d7584035d129bec4fde87accbabd2cea3bb784dfbbad: Status 404 returned error can't find the container with id db24675066306fa678e0d7584035d129bec4fde87accbabd2cea3bb784dfbbad Jul 21 12:40:23.279377 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:23.279339 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-kprgx"] Jul 21 12:40:23.283157 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:40:23.283121 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71802871_c712_4646_9b20_f6f78515bbc4.slice/crio-01127a049d2d008c4e3a6517ba9bf1fef7d7daaf1a86a168d3194c6263babfed WatchSource:0}: Error finding container 01127a049d2d008c4e3a6517ba9bf1fef7d7daaf1a86a168d3194c6263babfed: Status 404 returned error can't find the container with id 01127a049d2d008c4e3a6517ba9bf1fef7d7daaf1a86a168d3194c6263babfed Jul 21 12:40:23.574400 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:23.574311 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:23.574400 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:23.574376 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:23.574587 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:23.574544 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 12:40:23.574645 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:23.574632 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs podName:9bb8c882-1f66-4054-8f88-80a3fd392806 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:24.574612538 +0000 UTC m=+47.192905484 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs") pod "router-default-6974856856-hqzjg" (UID: "9bb8c882-1f66-4054-8f88-80a3fd392806") : secret "router-metrics-certs-default" not found Jul 21 12:40:23.574689 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:23.574651 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle podName:9bb8c882-1f66-4054-8f88-80a3fd392806 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:24.574638693 +0000 UTC m=+47.192931619 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle") pod "router-default-6974856856-hqzjg" (UID: "9bb8c882-1f66-4054-8f88-80a3fd392806") : configmap references non-existent config key: service-ca.crt Jul 21 12:40:24.145609 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:24.145560 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kprgx" event={"ID":"71802871-c712-4646-9b20-f6f78515bbc4","Type":"ContainerStarted","Data":"01127a049d2d008c4e3a6517ba9bf1fef7d7daaf1a86a168d3194c6263babfed"} Jul 21 12:40:24.147200 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:24.147169 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" event={"ID":"af138219-695f-4f5d-8a2f-df2c0a89f117","Type":"ContainerStarted","Data":"db24675066306fa678e0d7584035d129bec4fde87accbabd2cea3bb784dfbbad"} Jul 21 12:40:24.582306 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:24.582214 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:24.582546 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:24.582342 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:24.582546 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:24.582404 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 12:40:24.582546 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:24.582479 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs podName:9bb8c882-1f66-4054-8f88-80a3fd392806 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:26.582457668 +0000 UTC m=+49.200750598 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs") pod "router-default-6974856856-hqzjg" (UID: "9bb8c882-1f66-4054-8f88-80a3fd392806") : secret "router-metrics-certs-default" not found Jul 21 12:40:24.582546 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:24.582502 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle podName:9bb8c882-1f66-4054-8f88-80a3fd392806 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:26.582491187 +0000 UTC m=+49.200784120 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle") pod "router-default-6974856856-hqzjg" (UID: "9bb8c882-1f66-4054-8f88-80a3fd392806") : configmap references non-existent config key: service-ca.crt Jul 21 12:40:26.594728 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:26.594688 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:26.595112 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:26.594752 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:26.595112 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:26.594838 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 12:40:26.595112 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:26.594883 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs podName:9bb8c882-1f66-4054-8f88-80a3fd392806 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:30.594869519 +0000 UTC m=+53.213162446 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs") pod "router-default-6974856856-hqzjg" (UID: "9bb8c882-1f66-4054-8f88-80a3fd392806") : secret "router-metrics-certs-default" not found Jul 21 12:40:26.595112 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:26.594954 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle podName:9bb8c882-1f66-4054-8f88-80a3fd392806 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:30.594932289 +0000 UTC m=+53.213225215 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle") pod "router-default-6974856856-hqzjg" (UID: "9bb8c882-1f66-4054-8f88-80a3fd392806") : configmap references non-existent config key: service-ca.crt Jul 21 12:40:27.157332 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:27.157240 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kprgx" event={"ID":"71802871-c712-4646-9b20-f6f78515bbc4","Type":"ContainerStarted","Data":"fe19b06c1fb3484dfc43fa144871a8baa8c9ba0e618ca2564f7426019df98d0a"} Jul 21 12:40:27.158951 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:27.158926 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/0.log" Jul 21 12:40:27.159091 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:27.158960 2555 generic.go:356] "Generic (PLEG): container finished" podID="af138219-695f-4f5d-8a2f-df2c0a89f117" containerID="5924e1e247fc697780dfa5ce573c885a51589ef9940707d96b0b40952bcac4d0" exitCode=255 Jul 21 12:40:27.159091 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:27.159004 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" event={"ID":"af138219-695f-4f5d-8a2f-df2c0a89f117","Type":"ContainerDied","Data":"5924e1e247fc697780dfa5ce573c885a51589ef9940707d96b0b40952bcac4d0"} Jul 21 12:40:27.159224 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:27.159209 2555 scope.go:117] "RemoveContainer" containerID="5924e1e247fc697780dfa5ce573c885a51589ef9940707d96b0b40952bcac4d0" Jul 21 12:40:27.173909 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:27.173859 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kprgx" podStartSLOduration=1.545655623 podStartE2EDuration="5.173840248s" totalImagesPullingTime="3.628184625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:27.172631559 +0000 UTC m=+49.790924516" watchObservedRunningTime="2026-07-21 12:40:27.173840248 +0000 UTC m=+49.792133199" Jul 21 12:40:28.163319 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:28.163291 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/1.log" Jul 21 12:40:28.163759 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:28.163741 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/0.log" Jul 21 12:40:28.163818 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:28.163780 2555 generic.go:356] "Generic (PLEG): container finished" podID="af138219-695f-4f5d-8a2f-df2c0a89f117" containerID="6849b83a558de2ccb086a7be9575f50ddc754e8db08435423de5f90a65add5e3" exitCode=255 Jul 21 12:40:28.163905 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:28.163881 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" event={"ID":"af138219-695f-4f5d-8a2f-df2c0a89f117","Type":"ContainerDied","Data":"6849b83a558de2ccb086a7be9575f50ddc754e8db08435423de5f90a65add5e3"} Jul 21 12:40:28.163964 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:28.163946 2555 scope.go:117] "RemoveContainer" containerID="5924e1e247fc697780dfa5ce573c885a51589ef9940707d96b0b40952bcac4d0" Jul 21 12:40:28.164126 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:28.164111 2555 scope.go:117] "RemoveContainer" containerID="6849b83a558de2ccb086a7be9575f50ddc754e8db08435423de5f90a65add5e3" Jul 21 12:40:28.164296 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:28.164270 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-qnc6t_openshift-console-operator(af138219-695f-4f5d-8a2f-df2c0a89f117)\"" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" podUID="af138219-695f-4f5d-8a2f-df2c0a89f117" Jul 21 12:40:29.115265 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:29.115233 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls\") pod \"dns-default-gcm9k\" (UID: \"de273e85-f101-43f9-96ec-b7102d29ac49\") " pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:29.115477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:29.115301 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:29.115477 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:29.115400 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 12:40:29.115477 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:29.115403 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 12:40:29.115477 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:29.115475 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls podName:de273e85-f101-43f9-96ec-b7102d29ac49 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:45.115456333 +0000 UTC m=+67.733749290 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls") pod "dns-default-gcm9k" (UID: "de273e85-f101-43f9-96ec-b7102d29ac49") : secret "dns-default-metrics-tls" not found Jul 21 12:40:29.115671 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:29.115412 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fd754cdd7-bgzfh: secret "image-registry-tls" not found Jul 21 12:40:29.115671 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:29.115526 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls podName:b6e594b6-715f-4157-b411-94fbf7a3bdce nodeName:}" failed. No retries permitted until 2026-07-21 12:40:45.115514872 +0000 UTC m=+67.733807798 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls") pod "image-registry-7fd754cdd7-bgzfh" (UID: "b6e594b6-715f-4157-b411-94fbf7a3bdce") : secret "image-registry-tls" not found Jul 21 12:40:29.171548 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:29.171521 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/1.log" Jul 21 12:40:29.171929 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:29.171900 2555 scope.go:117] "RemoveContainer" containerID="6849b83a558de2ccb086a7be9575f50ddc754e8db08435423de5f90a65add5e3" Jul 21 12:40:29.172081 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:29.172061 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-qnc6t_openshift-console-operator(af138219-695f-4f5d-8a2f-df2c0a89f117)\"" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" podUID="af138219-695f-4f5d-8a2f-df2c0a89f117" Jul 21 12:40:30.626554 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.626507 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:30.627035 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.626589 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:30.627035 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:30.626717 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle podName:9bb8c882-1f66-4054-8f88-80a3fd392806 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:38.626696749 +0000 UTC m=+61.244989675 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle") pod "router-default-6974856856-hqzjg" (UID: "9bb8c882-1f66-4054-8f88-80a3fd392806") : configmap references non-existent config key: service-ca.crt Jul 21 12:40:30.627035 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:30.626726 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 12:40:30.627035 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:30.626762 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs podName:9bb8c882-1f66-4054-8f88-80a3fd392806 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:38.626751965 +0000 UTC m=+61.245044917 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs") pod "router-default-6974856856-hqzjg" (UID: "9bb8c882-1f66-4054-8f88-80a3fd392806") : secret "router-metrics-certs-default" not found Jul 21 12:40:30.749311 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.749276 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-v562r"] Jul 21 12:40:30.779989 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.779951 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v562r"] Jul 21 12:40:30.780140 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.780083 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:30.782120 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.782097 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 21 12:40:30.782347 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.782332 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 21 12:40:30.782690 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.782652 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6mtb2\"" Jul 21 12:40:30.782690 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.782677 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 21 12:40:30.782882 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.782676 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 21 12:40:30.828063 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.828022 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckk9c\" (UniqueName: \"kubernetes.io/projected/037a9656-a505-40ec-b7f8-03b73974213a-kube-api-access-ckk9c\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:30.828063 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.828071 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/037a9656-a505-40ec-b7f8-03b73974213a-crio-socket\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:30.828269 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.828132 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/037a9656-a505-40ec-b7f8-03b73974213a-data-volume\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:30.828269 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.828165 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:30.828269 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.828187 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/037a9656-a505-40ec-b7f8-03b73974213a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:30.929378 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.929290 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/037a9656-a505-40ec-b7f8-03b73974213a-data-volume\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:30.929378 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.929330 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:30.929378 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.929352 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/037a9656-a505-40ec-b7f8-03b73974213a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:30.929692 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.929428 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckk9c\" (UniqueName: \"kubernetes.io/projected/037a9656-a505-40ec-b7f8-03b73974213a-kube-api-access-ckk9c\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:30.929692 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.929453 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/037a9656-a505-40ec-b7f8-03b73974213a-crio-socket\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:30.929692 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:30.929465 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:40:30.929692 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:30.929514 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 12:40:30.929692 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:30.929541 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls podName:037a9656-a505-40ec-b7f8-03b73974213a nodeName:}" failed. No retries permitted until 2026-07-21 12:40:31.429519898 +0000 UTC m=+54.047812835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v562r" (UID: "037a9656-a505-40ec-b7f8-03b73974213a") : secret "insights-runtime-extractor-tls" not found Jul 21 12:40:30.929692 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:30.929574 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/037a9656-a505-40ec-b7f8-03b73974213a-kube-rbac-proxy-cm podName:037a9656-a505-40ec-b7f8-03b73974213a nodeName:}" failed. No retries permitted until 2026-07-21 12:40:31.429556244 +0000 UTC m=+54.047849197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/037a9656-a505-40ec-b7f8-03b73974213a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v562r" (UID: "037a9656-a505-40ec-b7f8-03b73974213a") : configmap "kube-rbac-proxy" not found Jul 21 12:40:30.929692 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.929650 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/037a9656-a505-40ec-b7f8-03b73974213a-crio-socket\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:30.930032 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.929703 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/037a9656-a505-40ec-b7f8-03b73974213a-data-volume\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:30.937613 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:30.937572 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckk9c\" (UniqueName: \"kubernetes.io/projected/037a9656-a505-40ec-b7f8-03b73974213a-kube-api-access-ckk9c\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:31.433703 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:31.433663 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:31.433703 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:31.433709 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/037a9656-a505-40ec-b7f8-03b73974213a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:31.433942 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:31.433792 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 12:40:31.433942 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:31.433841 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/037a9656-a505-40ec-b7f8-03b73974213a-kube-rbac-proxy-cm podName:037a9656-a505-40ec-b7f8-03b73974213a nodeName:}" failed. No retries permitted until 2026-07-21 12:40:32.433826866 +0000 UTC m=+55.052119791 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/037a9656-a505-40ec-b7f8-03b73974213a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v562r" (UID: "037a9656-a505-40ec-b7f8-03b73974213a") : configmap "kube-rbac-proxy" not found Jul 21 12:40:31.433942 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:31.433870 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:40:31.433942 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:31.433928 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls podName:037a9656-a505-40ec-b7f8-03b73974213a nodeName:}" failed. No retries permitted until 2026-07-21 12:40:32.433915176 +0000 UTC m=+55.052208102 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v562r" (UID: "037a9656-a505-40ec-b7f8-03b73974213a") : secret "insights-runtime-extractor-tls" not found Jul 21 12:40:32.442987 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:32.442942 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:32.442987 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:32.442990 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/037a9656-a505-40ec-b7f8-03b73974213a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:32.443515 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:32.443085 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 12:40:32.443515 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:32.443126 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:40:32.443515 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:32.443142 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/037a9656-a505-40ec-b7f8-03b73974213a-kube-rbac-proxy-cm podName:037a9656-a505-40ec-b7f8-03b73974213a nodeName:}" failed. No retries permitted until 2026-07-21 12:40:34.44312391 +0000 UTC m=+57.061416837 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/037a9656-a505-40ec-b7f8-03b73974213a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v562r" (UID: "037a9656-a505-40ec-b7f8-03b73974213a") : configmap "kube-rbac-proxy" not found Jul 21 12:40:32.443515 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:32.443213 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls podName:037a9656-a505-40ec-b7f8-03b73974213a nodeName:}" failed. No retries permitted until 2026-07-21 12:40:34.443192265 +0000 UTC m=+57.061485191 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v562r" (UID: "037a9656-a505-40ec-b7f8-03b73974213a") : secret "insights-runtime-extractor-tls" not found Jul 21 12:40:32.734758 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:32.734679 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-chngz_58089888-e584-4b42-ae32-e8074321df87/dns-node-resolver/0.log" Jul 21 12:40:33.009046 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:33.008945 2555 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:40:33.009046 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:33.009023 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:40:33.009424 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:33.009407 2555 scope.go:117] "RemoveContainer" containerID="6849b83a558de2ccb086a7be9575f50ddc754e8db08435423de5f90a65add5e3" Jul 21 12:40:33.009628 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:33.009575 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-qnc6t_openshift-console-operator(af138219-695f-4f5d-8a2f-df2c0a89f117)\"" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" podUID="af138219-695f-4f5d-8a2f-df2c0a89f117" Jul 21 12:40:33.934252 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:33.934220 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v98xb_ba7b78ab-59ae-485e-93cc-8005bd5b9a69/node-ca/0.log" Jul 21 12:40:34.459675 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:34.459623 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:34.459675 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:34.459678 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/037a9656-a505-40ec-b7f8-03b73974213a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:34.459948 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:34.459773 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:40:34.459948 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:34.459822 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 12:40:34.459948 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:34.459841 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls podName:037a9656-a505-40ec-b7f8-03b73974213a nodeName:}" failed. No retries permitted until 2026-07-21 12:40:38.459824618 +0000 UTC m=+61.078117543 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v562r" (UID: "037a9656-a505-40ec-b7f8-03b73974213a") : secret "insights-runtime-extractor-tls" not found Jul 21 12:40:34.459948 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:34.459882 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/037a9656-a505-40ec-b7f8-03b73974213a-kube-rbac-proxy-cm podName:037a9656-a505-40ec-b7f8-03b73974213a nodeName:}" failed. No retries permitted until 2026-07-21 12:40:38.45986434 +0000 UTC m=+61.078157269 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/037a9656-a505-40ec-b7f8-03b73974213a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v562r" (UID: "037a9656-a505-40ec-b7f8-03b73974213a") : configmap "kube-rbac-proxy" not found Jul 21 12:40:34.735927 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:34.735844 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f6p9g_02d7c116-6592-44ac-89f4-8cba0c30c18b/serve-healthcheck-canary/0.log" Jul 21 12:40:38.496462 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:38.496424 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:38.496945 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:38.496468 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/037a9656-a505-40ec-b7f8-03b73974213a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:38.496945 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:38.496569 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 12:40:38.496945 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:38.496570 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:40:38.496945 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:38.496657 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/037a9656-a505-40ec-b7f8-03b73974213a-kube-rbac-proxy-cm podName:037a9656-a505-40ec-b7f8-03b73974213a nodeName:}" failed. No retries permitted until 2026-07-21 12:40:46.496642748 +0000 UTC m=+69.114935674 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/037a9656-a505-40ec-b7f8-03b73974213a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v562r" (UID: "037a9656-a505-40ec-b7f8-03b73974213a") : configmap "kube-rbac-proxy" not found Jul 21 12:40:38.496945 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:38.496670 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls podName:037a9656-a505-40ec-b7f8-03b73974213a nodeName:}" failed. No retries permitted until 2026-07-21 12:40:46.496664324 +0000 UTC m=+69.114957250 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v562r" (UID: "037a9656-a505-40ec-b7f8-03b73974213a") : secret "insights-runtime-extractor-tls" not found Jul 21 12:40:38.698180 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:38.698141 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:38.698180 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:38.698192 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:38.698393 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:38.698316 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle podName:9bb8c882-1f66-4054-8f88-80a3fd392806 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:54.698301952 +0000 UTC m=+77.316594878 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle") pod "router-default-6974856856-hqzjg" (UID: "9bb8c882-1f66-4054-8f88-80a3fd392806") : configmap references non-existent config key: service-ca.crt Jul 21 12:40:38.698393 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:38.698315 2555 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 12:40:38.698393 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:38.698371 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs podName:9bb8c882-1f66-4054-8f88-80a3fd392806 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:54.698356983 +0000 UTC m=+77.316649911 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs") pod "router-default-6974856856-hqzjg" (UID: "9bb8c882-1f66-4054-8f88-80a3fd392806") : secret "router-metrics-certs-default" not found Jul 21 12:40:39.108000 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:39.107972 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ntfk8" Jul 21 12:40:43.640180 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:43.640143 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs\") pod \"network-metrics-daemon-z65kc\" (UID: \"eb0efdf9-1311-4257-9f40-3c35bee29d66\") " pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:40:43.642258 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:43.642240 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 12:40:43.651168 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:43.651147 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 21 12:40:43.651228 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:43.651216 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs podName:eb0efdf9-1311-4257-9f40-3c35bee29d66 nodeName:}" failed. No retries permitted until 2026-07-21 12:41:47.651199497 +0000 UTC m=+130.269492426 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs") pod "network-metrics-daemon-z65kc" (UID: "eb0efdf9-1311-4257-9f40-3c35bee29d66") : secret "metrics-daemon-secret" not found Jul 21 12:40:43.740507 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:43.740463 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret\") pod \"global-pull-secret-syncer-vtvm8\" (UID: \"a807b465-26c5-46ad-8abb-09a4f7716bee\") " pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:40:43.742617 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:43.742578 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 12:40:43.753862 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:43.753831 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a807b465-26c5-46ad-8abb-09a4f7716bee-original-pull-secret\") pod \"global-pull-secret-syncer-vtvm8\" (UID: \"a807b465-26c5-46ad-8abb-09a4f7716bee\") " pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:40:43.841435 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:43.841393 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lvlz\" (UniqueName: \"kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz\") pod \"network-check-target-jblzc\" (UID: \"2026fc18-7ca0-4d3e-8adf-aca525a12986\") " pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:40:43.844259 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:43.844225 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lvlz\" (UniqueName: \"kubernetes.io/projected/2026fc18-7ca0-4d3e-8adf-aca525a12986-kube-api-access-7lvlz\") pod \"network-check-target-jblzc\" (UID: \"2026fc18-7ca0-4d3e-8adf-aca525a12986\") " pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:40:44.030988 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:44.030908 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vtvm8" Jul 21 12:40:44.043154 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:44.043131 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rhfvk\"" Jul 21 12:40:44.051952 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:44.051922 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:40:44.172368 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:44.172270 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vtvm8"] Jul 21 12:40:44.177038 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:40:44.177013 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda807b465_26c5_46ad_8abb_09a4f7716bee.slice/crio-8bb7cf0705bbff4c2deba01234855dfe317ed8695bc5e8ca1d43841b3fe31247 WatchSource:0}: Error finding container 8bb7cf0705bbff4c2deba01234855dfe317ed8695bc5e8ca1d43841b3fe31247: Status 404 returned error can't find the container with id 8bb7cf0705bbff4c2deba01234855dfe317ed8695bc5e8ca1d43841b3fe31247 Jul 21 12:40:44.194933 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:44.194902 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jblzc"] Jul 21 12:40:44.198933 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:40:44.198910 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2026fc18_7ca0_4d3e_8adf_aca525a12986.slice/crio-a42a7b80572592eb5b39c6e8059c89895290f398beb2d77f239262555dc81f01 WatchSource:0}: Error finding container a42a7b80572592eb5b39c6e8059c89895290f398beb2d77f239262555dc81f01: Status 404 returned error can't find the container with id a42a7b80572592eb5b39c6e8059c89895290f398beb2d77f239262555dc81f01 Jul 21 12:40:44.208850 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:44.208815 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jblzc" event={"ID":"2026fc18-7ca0-4d3e-8adf-aca525a12986","Type":"ContainerStarted","Data":"a42a7b80572592eb5b39c6e8059c89895290f398beb2d77f239262555dc81f01"} Jul 21 12:40:44.209818 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:44.209794 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vtvm8" event={"ID":"a807b465-26c5-46ad-8abb-09a4f7716bee","Type":"ContainerStarted","Data":"8bb7cf0705bbff4c2deba01234855dfe317ed8695bc5e8ca1d43841b3fe31247"} Jul 21 12:40:44.920006 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:44.919974 2555 scope.go:117] "RemoveContainer" containerID="6849b83a558de2ccb086a7be9575f50ddc754e8db08435423de5f90a65add5e3" Jul 21 12:40:45.154424 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:45.154388 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:45.154630 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:45.154454 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls\") pod \"dns-default-gcm9k\" (UID: \"de273e85-f101-43f9-96ec-b7102d29ac49\") " pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:45.156926 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:45.156890 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de273e85-f101-43f9-96ec-b7102d29ac49-metrics-tls\") pod \"dns-default-gcm9k\" (UID: \"de273e85-f101-43f9-96ec-b7102d29ac49\") " pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:45.156926 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:45.156897 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls\") pod \"image-registry-7fd754cdd7-bgzfh\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:45.215836 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:45.215719 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jblzc" event={"ID":"2026fc18-7ca0-4d3e-8adf-aca525a12986","Type":"ContainerStarted","Data":"ee890e283357085948a84b847db858129d5c13a9ea4888ef83828503be10549c"} Jul 21 12:40:45.215836 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:45.215813 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:40:45.217548 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:45.217528 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 12:40:45.218043 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:45.218029 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/1.log" Jul 21 12:40:45.218113 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:45.218067 2555 generic.go:356] "Generic (PLEG): container finished" podID="af138219-695f-4f5d-8a2f-df2c0a89f117" containerID="97c6110c0f799066aa523f415b46b160a81bd17798b2a547235bf82866add7ab" exitCode=255 Jul 21 12:40:45.218160 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:45.218113 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" event={"ID":"af138219-695f-4f5d-8a2f-df2c0a89f117","Type":"ContainerDied","Data":"97c6110c0f799066aa523f415b46b160a81bd17798b2a547235bf82866add7ab"} Jul 21 12:40:45.218203 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:45.218162 2555 scope.go:117] "RemoveContainer" containerID="6849b83a558de2ccb086a7be9575f50ddc754e8db08435423de5f90a65add5e3" Jul 21 12:40:45.218504 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:45.218488 2555 scope.go:117] "RemoveContainer" containerID="97c6110c0f799066aa523f415b46b160a81bd17798b2a547235bf82866add7ab" Jul 21 12:40:45.218707 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:45.218690 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-qnc6t_openshift-console-operator(af138219-695f-4f5d-8a2f-df2c0a89f117)\"" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" podUID="af138219-695f-4f5d-8a2f-df2c0a89f117" Jul 21 12:40:45.250194 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:45.250135 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jblzc" podStartSLOduration=67.25011593 podStartE2EDuration="1m7.25011593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:39:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:45.231448044 +0000 UTC m=+67.849740993" watchObservedRunningTime="2026-07-21 12:40:45.25011593 +0000 UTC m=+67.868408879" Jul 21 12:40:45.331747 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:45.331711 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-v4mnz\"" Jul 21 12:40:45.339750 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:45.339721 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:45.352315 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:45.352298 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h5b4t\"" Jul 21 12:40:45.360367 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:45.360345 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:45.507658 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:45.507623 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fd754cdd7-bgzfh"] Jul 21 12:40:45.513032 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:40:45.512395 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6e594b6_715f_4157_b411_94fbf7a3bdce.slice/crio-9faca21b86d1e62bf1b23cad6e88e47d4b3591f4d8acc664d88ed9f8fcc6d7d1 WatchSource:0}: Error finding container 9faca21b86d1e62bf1b23cad6e88e47d4b3591f4d8acc664d88ed9f8fcc6d7d1: Status 404 returned error can't find the container with id 9faca21b86d1e62bf1b23cad6e88e47d4b3591f4d8acc664d88ed9f8fcc6d7d1 Jul 21 12:40:45.536935 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:45.535297 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gcm9k"] Jul 21 12:40:45.541349 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:40:45.541307 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde273e85_f101_43f9_96ec_b7102d29ac49.slice/crio-ffe3bfb2751860e103131c8a36eb0fae7000e002cfda8c444c199375631e0ee0 WatchSource:0}: Error finding container ffe3bfb2751860e103131c8a36eb0fae7000e002cfda8c444c199375631e0ee0: Status 404 returned error can't find the container with id ffe3bfb2751860e103131c8a36eb0fae7000e002cfda8c444c199375631e0ee0 Jul 21 12:40:46.224375 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:46.224331 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" event={"ID":"b6e594b6-715f-4157-b411-94fbf7a3bdce","Type":"ContainerStarted","Data":"803052ce486ada3570a56ea40e8b7b4b29a982a918c6798d4491ac815fb7e14f"} Jul 21 12:40:46.224864 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:46.224400 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" event={"ID":"b6e594b6-715f-4157-b411-94fbf7a3bdce","Type":"ContainerStarted","Data":"9faca21b86d1e62bf1b23cad6e88e47d4b3591f4d8acc664d88ed9f8fcc6d7d1"} Jul 21 12:40:46.224864 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:46.224426 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:40:46.226700 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:46.226675 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 12:40:46.228089 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:46.228063 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gcm9k" event={"ID":"de273e85-f101-43f9-96ec-b7102d29ac49","Type":"ContainerStarted","Data":"ffe3bfb2751860e103131c8a36eb0fae7000e002cfda8c444c199375631e0ee0"} Jul 21 12:40:46.246058 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:46.245989 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" podStartSLOduration=68.245967478 podStartE2EDuration="1m8.245967478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:39:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:46.245835718 +0000 UTC m=+68.864128671" watchObservedRunningTime="2026-07-21 12:40:46.245967478 +0000 UTC m=+68.864260426" Jul 21 12:40:46.569353 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:46.569266 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:46.569353 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:46.569308 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/037a9656-a505-40ec-b7f8-03b73974213a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:46.569585 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:46.569431 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 12:40:46.569585 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:46.569510 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls podName:037a9656-a505-40ec-b7f8-03b73974213a nodeName:}" failed. No retries permitted until 2026-07-21 12:41:02.569489844 +0000 UTC m=+85.187782796 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v562r" (UID: "037a9656-a505-40ec-b7f8-03b73974213a") : secret "insights-runtime-extractor-tls" not found Jul 21 12:40:46.569852 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:46.569835 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/037a9656-a505-40ec-b7f8-03b73974213a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:40:49.243671 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:49.243628 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gcm9k" event={"ID":"de273e85-f101-43f9-96ec-b7102d29ac49","Type":"ContainerStarted","Data":"09b7ed0608287dee410809881087a829650b4858165d53052ef036dd367edbce"} Jul 21 12:40:49.244188 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:49.243740 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gcm9k" event={"ID":"de273e85-f101-43f9-96ec-b7102d29ac49","Type":"ContainerStarted","Data":"4bdc6f42eea9f80285876eb409c0b0184e34cde560daafb7f9dde1f0ae07b726"} Jul 21 12:40:49.244188 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:49.243779 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:49.245075 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:49.245051 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vtvm8" event={"ID":"a807b465-26c5-46ad-8abb-09a4f7716bee","Type":"ContainerStarted","Data":"536f0dd617ad57d1277e8c2a9af4cebd3e3890390224d370306a113da3bd45d5"} Jul 21 12:40:49.262518 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:49.262468 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gcm9k" podStartSLOduration=33.014938114 podStartE2EDuration="36.262454201s" totalImagesPullingTime="3.247516087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:49.261738111 +0000 UTC m=+71.880031059" watchObservedRunningTime="2026-07-21 12:40:49.262454201 +0000 UTC m=+71.880747148" Jul 21 12:40:49.277312 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:49.277267 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vtvm8" podStartSLOduration=66.65517472 podStartE2EDuration="1m11.277251658s" totalImagesPullingTime="4.622076938s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:49.276547629 +0000 UTC m=+71.894840578" watchObservedRunningTime="2026-07-21 12:40:49.277251658 +0000 UTC m=+71.895544606" Jul 21 12:40:53.008912 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:53.008881 2555 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:40:53.009286 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:53.008932 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:40:53.009286 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:53.009230 2555 scope.go:117] "RemoveContainer" containerID="97c6110c0f799066aa523f415b46b160a81bd17798b2a547235bf82866add7ab" Jul 21 12:40:53.009389 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:40:53.009370 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-qnc6t_openshift-console-operator(af138219-695f-4f5d-8a2f-df2c0a89f117)\"" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" podUID="af138219-695f-4f5d-8a2f-df2c0a89f117" Jul 21 12:40:54.732351 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:54.732293 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:54.732351 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:54.732356 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:54.732984 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:54.732967 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9bb8c882-1f66-4054-8f88-80a3fd392806-service-ca-bundle\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:54.734773 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:54.734755 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9bb8c882-1f66-4054-8f88-80a3fd392806-metrics-certs\") pod \"router-default-6974856856-hqzjg\" (UID: \"9bb8c882-1f66-4054-8f88-80a3fd392806\") " pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:54.929059 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:54.929023 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-drcb2\"" Jul 21 12:40:54.936304 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:54.936282 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:55.076502 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:55.076434 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6974856856-hqzjg"] Jul 21 12:40:55.079187 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:40:55.079157 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bb8c882_1f66_4054_8f88_80a3fd392806.slice/crio-fb0b561ed6741098814983c7fe075fa1432492aba5deee7ae5499e57757a0921 WatchSource:0}: Error finding container fb0b561ed6741098814983c7fe075fa1432492aba5deee7ae5499e57757a0921: Status 404 returned error can't find the container with id fb0b561ed6741098814983c7fe075fa1432492aba5deee7ae5499e57757a0921 Jul 21 12:40:55.267471 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:55.267386 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6974856856-hqzjg" event={"ID":"9bb8c882-1f66-4054-8f88-80a3fd392806","Type":"ContainerStarted","Data":"baf00bf951606fe169934efcdae89697c84f97485d4d4b166ac8b5a050706efd"} Jul 21 12:40:55.267471 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:55.267436 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6974856856-hqzjg" event={"ID":"9bb8c882-1f66-4054-8f88-80a3fd392806","Type":"ContainerStarted","Data":"fb0b561ed6741098814983c7fe075fa1432492aba5deee7ae5499e57757a0921"} Jul 21 12:40:55.293409 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:55.293356 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6974856856-hqzjg" podStartSLOduration=33.293338354 podStartE2EDuration="33.293338354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:55.292074669 +0000 UTC m=+77.910367616" watchObservedRunningTime="2026-07-21 12:40:55.293338354 +0000 UTC m=+77.911631301" Jul 21 12:40:55.936953 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:55.936919 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:55.939563 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:55.939536 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:56.270657 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:56.270556 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:56.271918 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:56.271898 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6974856856-hqzjg" Jul 21 12:40:59.250859 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.250726 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gcm9k" Jul 21 12:40:59.397153 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.397109 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-b89bf5568-qf6j8"] Jul 21 12:40:59.415352 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.415316 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-b89bf5568-qf6j8"] Jul 21 12:40:59.415495 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.415411 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b89bf5568-qf6j8" Jul 21 12:40:59.418467 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.418409 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 21 12:40:59.418694 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.418537 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 21 12:40:59.418778 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.418714 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 21 12:40:59.418857 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.418786 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 21 12:40:59.477223 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.477188 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-tcq6l"] Jul 21 12:40:59.489755 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.489726 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-tcq6l" Jul 21 12:40:59.491886 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.491851 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 21 12:40:59.491886 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.491876 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-272wz\"" Jul 21 12:40:59.492079 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.491981 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 21 12:40:59.499952 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.499922 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-tcq6l"] Jul 21 12:40:59.571241 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.571146 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/96c83e06-37e1-4bb8-82ab-3db5462a7528-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-tcq6l\" (UID: \"96c83e06-37e1-4bb8-82ab-3db5462a7528\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-tcq6l" Jul 21 12:40:59.571241 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.571187 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgrfh\" (UniqueName: \"kubernetes.io/projected/1c5f930d-0e27-4a08-b34a-d1579855db94-kube-api-access-jgrfh\") pod \"klusterlet-addon-workmgr-b89bf5568-qf6j8\" (UID: \"1c5f930d-0e27-4a08-b34a-d1579855db94\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b89bf5568-qf6j8" Jul 21 12:40:59.571241 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.571210 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/96c83e06-37e1-4bb8-82ab-3db5462a7528-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-tcq6l\" (UID: \"96c83e06-37e1-4bb8-82ab-3db5462a7528\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-tcq6l" Jul 21 12:40:59.571495 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.571318 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c5f930d-0e27-4a08-b34a-d1579855db94-tmp\") pod \"klusterlet-addon-workmgr-b89bf5568-qf6j8\" (UID: \"1c5f930d-0e27-4a08-b34a-d1579855db94\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b89bf5568-qf6j8" Jul 21 12:40:59.571495 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.571379 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1c5f930d-0e27-4a08-b34a-d1579855db94-klusterlet-config\") pod \"klusterlet-addon-workmgr-b89bf5568-qf6j8\" (UID: \"1c5f930d-0e27-4a08-b34a-d1579855db94\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b89bf5568-qf6j8" Jul 21 12:40:59.672510 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.672471 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1c5f930d-0e27-4a08-b34a-d1579855db94-klusterlet-config\") pod \"klusterlet-addon-workmgr-b89bf5568-qf6j8\" (UID: \"1c5f930d-0e27-4a08-b34a-d1579855db94\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b89bf5568-qf6j8" Jul 21 12:40:59.672510 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.672517 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/96c83e06-37e1-4bb8-82ab-3db5462a7528-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-tcq6l\" (UID: \"96c83e06-37e1-4bb8-82ab-3db5462a7528\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-tcq6l" Jul 21 12:40:59.672792 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.672541 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgrfh\" (UniqueName: \"kubernetes.io/projected/1c5f930d-0e27-4a08-b34a-d1579855db94-kube-api-access-jgrfh\") pod \"klusterlet-addon-workmgr-b89bf5568-qf6j8\" (UID: \"1c5f930d-0e27-4a08-b34a-d1579855db94\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b89bf5568-qf6j8" Jul 21 12:40:59.672792 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.672562 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/96c83e06-37e1-4bb8-82ab-3db5462a7528-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-tcq6l\" (UID: \"96c83e06-37e1-4bb8-82ab-3db5462a7528\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-tcq6l" Jul 21 12:40:59.672792 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.672633 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c5f930d-0e27-4a08-b34a-d1579855db94-tmp\") pod \"klusterlet-addon-workmgr-b89bf5568-qf6j8\" (UID: \"1c5f930d-0e27-4a08-b34a-d1579855db94\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b89bf5568-qf6j8" Jul 21 12:40:59.673024 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.673004 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c5f930d-0e27-4a08-b34a-d1579855db94-tmp\") pod \"klusterlet-addon-workmgr-b89bf5568-qf6j8\" (UID: \"1c5f930d-0e27-4a08-b34a-d1579855db94\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b89bf5568-qf6j8" Jul 21 12:40:59.673318 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.673295 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/96c83e06-37e1-4bb8-82ab-3db5462a7528-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-tcq6l\" (UID: \"96c83e06-37e1-4bb8-82ab-3db5462a7528\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-tcq6l" Jul 21 12:40:59.675040 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.675021 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/96c83e06-37e1-4bb8-82ab-3db5462a7528-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-tcq6l\" (UID: \"96c83e06-37e1-4bb8-82ab-3db5462a7528\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-tcq6l" Jul 21 12:40:59.675166 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.675147 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1c5f930d-0e27-4a08-b34a-d1579855db94-klusterlet-config\") pod \"klusterlet-addon-workmgr-b89bf5568-qf6j8\" (UID: \"1c5f930d-0e27-4a08-b34a-d1579855db94\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b89bf5568-qf6j8" Jul 21 12:40:59.681883 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.681860 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgrfh\" (UniqueName: \"kubernetes.io/projected/1c5f930d-0e27-4a08-b34a-d1579855db94-kube-api-access-jgrfh\") pod \"klusterlet-addon-workmgr-b89bf5568-qf6j8\" (UID: \"1c5f930d-0e27-4a08-b34a-d1579855db94\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b89bf5568-qf6j8" Jul 21 12:40:59.725677 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.725633 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b89bf5568-qf6j8" Jul 21 12:40:59.798720 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.798684 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-tcq6l" Jul 21 12:40:59.878718 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.878682 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-b89bf5568-qf6j8"] Jul 21 12:40:59.882067 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:40:59.882041 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c5f930d_0e27_4a08_b34a_d1579855db94.slice/crio-4cac597f66facda0c6f950d934742e072f6e015f1096a6287e41f4f079277178 WatchSource:0}: Error finding container 4cac597f66facda0c6f950d934742e072f6e015f1096a6287e41f4f079277178: Status 404 returned error can't find the container with id 4cac597f66facda0c6f950d934742e072f6e015f1096a6287e41f4f079277178 Jul 21 12:40:59.940831 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:40:59.940804 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-tcq6l"] Jul 21 12:40:59.944042 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:40:59.944013 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96c83e06_37e1_4bb8_82ab_3db5462a7528.slice/crio-b77f52d5cacad2dc38620fd637e504ff57efd55071b839d0451d992c2ace6fcc WatchSource:0}: Error finding container b77f52d5cacad2dc38620fd637e504ff57efd55071b839d0451d992c2ace6fcc: Status 404 returned error can't find the container with id b77f52d5cacad2dc38620fd637e504ff57efd55071b839d0451d992c2ace6fcc Jul 21 12:41:00.283798 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:00.283707 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-tcq6l" event={"ID":"96c83e06-37e1-4bb8-82ab-3db5462a7528","Type":"ContainerStarted","Data":"b77f52d5cacad2dc38620fd637e504ff57efd55071b839d0451d992c2ace6fcc"} Jul 21 12:41:00.284749 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:00.284728 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b89bf5568-qf6j8" event={"ID":"1c5f930d-0e27-4a08-b34a-d1579855db94","Type":"ContainerStarted","Data":"4cac597f66facda0c6f950d934742e072f6e015f1096a6287e41f4f079277178"} Jul 21 12:41:01.833629 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:01.832524 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-72bwk"] Jul 21 12:41:01.839498 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:01.838772 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-72bwk" Jul 21 12:41:01.840852 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:01.840828 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 21 12:41:01.840974 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:01.840828 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-cb5vz\"" Jul 21 12:41:01.844781 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:01.844754 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-72bwk"] Jul 21 12:41:01.995119 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:01.995081 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e1e07274-8366-4ce0-83f1-9994b75b963b-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-72bwk\" (UID: \"e1e07274-8366-4ce0-83f1-9994b75b963b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-72bwk" Jul 21 12:41:02.096362 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:02.096320 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e1e07274-8366-4ce0-83f1-9994b75b963b-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-72bwk\" (UID: \"e1e07274-8366-4ce0-83f1-9994b75b963b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-72bwk" Jul 21 12:41:02.099537 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:02.099501 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e1e07274-8366-4ce0-83f1-9994b75b963b-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-72bwk\" (UID: \"e1e07274-8366-4ce0-83f1-9994b75b963b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-72bwk" Jul 21 12:41:02.152493 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:02.152453 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-72bwk" Jul 21 12:41:02.400964 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:02.400891 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-72bwk"] Jul 21 12:41:02.603961 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:02.603917 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:41:02.606565 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:02.606538 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/037a9656-a505-40ec-b7f8-03b73974213a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v562r\" (UID: \"037a9656-a505-40ec-b7f8-03b73974213a\") " pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:41:02.891163 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:02.891130 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6mtb2\"" Jul 21 12:41:02.900014 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:02.899977 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v562r" Jul 21 12:41:03.303557 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:03.303461 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-tcq6l" event={"ID":"96c83e06-37e1-4bb8-82ab-3db5462a7528","Type":"ContainerStarted","Data":"c5432f527fa00a24ba619e020ccabf530908f086727b7dd9a9c500f9b288261f"} Jul 21 12:41:03.321178 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:03.321115 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-tcq6l" podStartSLOduration=2.012243908 podStartE2EDuration="4.321092439s" totalImagesPullingTime="2.308848531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:03.319894613 +0000 UTC m=+85.938187561" watchObservedRunningTime="2026-07-21 12:41:03.321092439 +0000 UTC m=+85.939385391" Jul 21 12:41:03.535475 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:41:03.535445 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1e07274_8366_4ce0_83f1_9994b75b963b.slice/crio-34a48989297862f8af89f63ccd973b256181dc99201abed8ae4336bc68125dcd WatchSource:0}: Error finding container 34a48989297862f8af89f63ccd973b256181dc99201abed8ae4336bc68125dcd: Status 404 returned error can't find the container with id 34a48989297862f8af89f63ccd973b256181dc99201abed8ae4336bc68125dcd Jul 21 12:41:03.672095 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:03.671809 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v562r"] Jul 21 12:41:03.676265 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:41:03.676233 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod037a9656_a505_40ec_b7f8_03b73974213a.slice/crio-502215bd425902ac9d1ae4c88affae3cb773bf8a704847b569de4e469034282c WatchSource:0}: Error finding container 502215bd425902ac9d1ae4c88affae3cb773bf8a704847b569de4e469034282c: Status 404 returned error can't find the container with id 502215bd425902ac9d1ae4c88affae3cb773bf8a704847b569de4e469034282c Jul 21 12:41:04.310061 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:04.310012 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v562r" event={"ID":"037a9656-a505-40ec-b7f8-03b73974213a","Type":"ContainerStarted","Data":"502215bd425902ac9d1ae4c88affae3cb773bf8a704847b569de4e469034282c"} Jul 21 12:41:04.311939 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:04.311908 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-72bwk" event={"ID":"e1e07274-8366-4ce0-83f1-9994b75b963b","Type":"ContainerStarted","Data":"34a48989297862f8af89f63ccd973b256181dc99201abed8ae4336bc68125dcd"} Jul 21 12:41:04.314516 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:04.314481 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b89bf5568-qf6j8" event={"ID":"1c5f930d-0e27-4a08-b34a-d1579855db94","Type":"ContainerStarted","Data":"4b76685eb8856d7357b1e3c73dabb7817efa16bbd5fa78eab297f203e6bdc8c5"} Jul 21 12:41:04.314749 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:04.314708 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b89bf5568-qf6j8" Jul 21 12:41:04.316657 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:04.316612 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b89bf5568-qf6j8" Jul 21 12:41:04.334879 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:04.334697 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-b89bf5568-qf6j8" podStartSLOduration=1.640667411 podStartE2EDuration="5.334676133s" totalImagesPullingTime="3.694008722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:04.333943153 +0000 UTC m=+86.952236102" watchObservedRunningTime="2026-07-21 12:41:04.334676133 +0000 UTC m=+86.952969082" Jul 21 12:41:04.920033 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:04.920002 2555 scope.go:117] "RemoveContainer" containerID="97c6110c0f799066aa523f415b46b160a81bd17798b2a547235bf82866add7ab" Jul 21 12:41:04.920326 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:41:04.920187 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-qnc6t_openshift-console-operator(af138219-695f-4f5d-8a2f-df2c0a89f117)\"" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" podUID="af138219-695f-4f5d-8a2f-df2c0a89f117" Jul 21 12:41:05.346581 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:05.346526 2555 patch_prober.go:28] interesting pod/image-registry-7fd754cdd7-bgzfh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 21 12:41:05.347146 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:05.346610 2555 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" podUID="b6e594b6-715f-4157-b411-94fbf7a3bdce" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 21 12:41:06.324042 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:06.324004 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-72bwk" event={"ID":"e1e07274-8366-4ce0-83f1-9994b75b963b","Type":"ContainerStarted","Data":"e8775517b02a14b1d1defe7f826aa199e5785ac5f6a5795a21dfc59044cd380b"} Jul 21 12:41:06.343808 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:06.343735 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-72bwk" podStartSLOduration=3.372922 podStartE2EDuration="5.343715299s" totalImagesPullingTime="1.970793299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:41:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:06.342644551 +0000 UTC m=+88.960937502" watchObservedRunningTime="2026-07-21 12:41:06.343715299 +0000 UTC m=+88.962008225" Jul 21 12:41:07.237540 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:07.237509 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:41:07.328203 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:07.328167 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-72bwk" Jul 21 12:41:07.334107 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:07.334078 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-72bwk" Jul 21 12:41:07.907144 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:07.907108 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d"] Jul 21 12:41:07.909044 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:07.909026 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d" Jul 21 12:41:07.912805 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:07.912555 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 21 12:41:07.912805 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:07.912762 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 21 12:41:07.915171 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:07.915144 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 21 12:41:07.915344 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:07.915187 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 21 12:41:07.915434 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:07.915416 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 21 12:41:07.915507 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:07.915416 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-4wksk\"" Jul 21 12:41:07.924248 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:07.924225 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d"] Jul 21 12:41:08.055652 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:08.055584 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf7nj\" (UniqueName: \"kubernetes.io/projected/c4a34804-6773-4cb3-8abe-8e1a965e23e2-kube-api-access-cf7nj\") pod \"prometheus-operator-5945b5cfd-lhv2d\" (UID: \"c4a34804-6773-4cb3-8abe-8e1a965e23e2\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d" Jul 21 12:41:08.055652 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:08.055655 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4a34804-6773-4cb3-8abe-8e1a965e23e2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-lhv2d\" (UID: \"c4a34804-6773-4cb3-8abe-8e1a965e23e2\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d" Jul 21 12:41:08.055873 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:08.055762 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4a34804-6773-4cb3-8abe-8e1a965e23e2-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-lhv2d\" (UID: \"c4a34804-6773-4cb3-8abe-8e1a965e23e2\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d" Jul 21 12:41:08.055873 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:08.055805 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4a34804-6773-4cb3-8abe-8e1a965e23e2-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-lhv2d\" (UID: \"c4a34804-6773-4cb3-8abe-8e1a965e23e2\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d" Jul 21 12:41:08.156299 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:08.156260 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf7nj\" (UniqueName: \"kubernetes.io/projected/c4a34804-6773-4cb3-8abe-8e1a965e23e2-kube-api-access-cf7nj\") pod \"prometheus-operator-5945b5cfd-lhv2d\" (UID: \"c4a34804-6773-4cb3-8abe-8e1a965e23e2\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d" Jul 21 12:41:08.156494 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:08.156328 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4a34804-6773-4cb3-8abe-8e1a965e23e2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-lhv2d\" (UID: \"c4a34804-6773-4cb3-8abe-8e1a965e23e2\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d" Jul 21 12:41:08.156494 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:08.156370 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4a34804-6773-4cb3-8abe-8e1a965e23e2-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-lhv2d\" (UID: \"c4a34804-6773-4cb3-8abe-8e1a965e23e2\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d" Jul 21 12:41:08.156494 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:08.156398 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4a34804-6773-4cb3-8abe-8e1a965e23e2-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-lhv2d\" (UID: \"c4a34804-6773-4cb3-8abe-8e1a965e23e2\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d" Jul 21 12:41:08.157258 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:08.157174 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4a34804-6773-4cb3-8abe-8e1a965e23e2-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-lhv2d\" (UID: \"c4a34804-6773-4cb3-8abe-8e1a965e23e2\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d" Jul 21 12:41:08.159176 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:08.159151 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4a34804-6773-4cb3-8abe-8e1a965e23e2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-lhv2d\" (UID: \"c4a34804-6773-4cb3-8abe-8e1a965e23e2\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d" Jul 21 12:41:08.159314 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:08.159160 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4a34804-6773-4cb3-8abe-8e1a965e23e2-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-lhv2d\" (UID: \"c4a34804-6773-4cb3-8abe-8e1a965e23e2\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d" Jul 21 12:41:08.165914 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:08.165861 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf7nj\" (UniqueName: \"kubernetes.io/projected/c4a34804-6773-4cb3-8abe-8e1a965e23e2-kube-api-access-cf7nj\") pod \"prometheus-operator-5945b5cfd-lhv2d\" (UID: \"c4a34804-6773-4cb3-8abe-8e1a965e23e2\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d" Jul 21 12:41:08.221855 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:08.221813 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d" Jul 21 12:41:08.754017 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:08.753801 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d"] Jul 21 12:41:08.756759 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:41:08.756732 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4a34804_6773_4cb3_8abe_8e1a965e23e2.slice/crio-fadb09664623bbe614a8d1ea0f6ac6b10d2cb09cba0806ecb4994ca4250a6ba4 WatchSource:0}: Error finding container fadb09664623bbe614a8d1ea0f6ac6b10d2cb09cba0806ecb4994ca4250a6ba4: Status 404 returned error can't find the container with id fadb09664623bbe614a8d1ea0f6ac6b10d2cb09cba0806ecb4994ca4250a6ba4 Jul 21 12:41:09.337345 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:09.337308 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v562r" event={"ID":"037a9656-a505-40ec-b7f8-03b73974213a","Type":"ContainerStarted","Data":"d5767adf92930d1a26a0a402f21f3cd41405f195963074f724415ddbda091b9d"} Jul 21 12:41:09.339222 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:09.339193 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d" event={"ID":"c4a34804-6773-4cb3-8abe-8e1a965e23e2","Type":"ContainerStarted","Data":"fadb09664623bbe614a8d1ea0f6ac6b10d2cb09cba0806ecb4994ca4250a6ba4"} Jul 21 12:41:12.355479 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:12.355448 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v562r" event={"ID":"037a9656-a505-40ec-b7f8-03b73974213a","Type":"ContainerStarted","Data":"174ba4c112e43b79f6ca04af98d721b644aee981097c64f4636ec34094c985d7"} Jul 21 12:41:12.357153 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:12.357109 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d" event={"ID":"c4a34804-6773-4cb3-8abe-8e1a965e23e2","Type":"ContainerStarted","Data":"275903c51c415d4110865f7215d7b78d35f4602dc8792cd7e29d92f87cfc3275"} Jul 21 12:41:13.364255 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:13.364221 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d" event={"ID":"c4a34804-6773-4cb3-8abe-8e1a965e23e2","Type":"ContainerStarted","Data":"bd45b7508736ae706f02394f4249d2f50bc25a061a22fdffdce95af9f761de25"} Jul 21 12:41:13.387078 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:13.387015 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-lhv2d" podStartSLOduration=2.915357685 podStartE2EDuration="6.386992112s" totalImagesPullingTime="3.471634427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:13.386058334 +0000 UTC m=+96.004351283" watchObservedRunningTime="2026-07-21 12:41:13.386992112 +0000 UTC m=+96.005285061" Jul 21 12:41:15.290193 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.290155 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr"] Jul 21 12:41:15.305285 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.305250 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" Jul 21 12:41:15.306292 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.306253 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr"] Jul 21 12:41:15.307579 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.307552 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-z5zbm\"" Jul 21 12:41:15.309179 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.309160 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 21 12:41:15.309849 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.309826 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 21 12:41:15.316769 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.316746 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/631f4a0a-8dab-49f8-a4da-b493b6d1343a-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-l72lr\" (UID: \"631f4a0a-8dab-49f8-a4da-b493b6d1343a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" Jul 21 12:41:15.316899 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.316781 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/631f4a0a-8dab-49f8-a4da-b493b6d1343a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-l72lr\" (UID: \"631f4a0a-8dab-49f8-a4da-b493b6d1343a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" Jul 21 12:41:15.316899 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.316804 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/631f4a0a-8dab-49f8-a4da-b493b6d1343a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-l72lr\" (UID: \"631f4a0a-8dab-49f8-a4da-b493b6d1343a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" Jul 21 12:41:15.316899 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.316860 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svlrs\" (UniqueName: \"kubernetes.io/projected/631f4a0a-8dab-49f8-a4da-b493b6d1343a-kube-api-access-svlrs\") pod \"openshift-state-metrics-79f6958cdf-l72lr\" (UID: \"631f4a0a-8dab-49f8-a4da-b493b6d1343a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" Jul 21 12:41:15.325248 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.322807 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-d4t5z"] Jul 21 12:41:15.326054 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.326003 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.329311 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.329285 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 21 12:41:15.329451 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.329337 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 21 12:41:15.329669 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.329642 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nj2hc\"" Jul 21 12:41:15.329789 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.329715 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 21 12:41:15.417658 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.417626 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/631f4a0a-8dab-49f8-a4da-b493b6d1343a-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-l72lr\" (UID: \"631f4a0a-8dab-49f8-a4da-b493b6d1343a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" Jul 21 12:41:15.417858 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.417665 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0dd584cd-f447-4f04-bc66-efb40e36578f-node-exporter-wtmp\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.417858 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.417699 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0dd584cd-f447-4f04-bc66-efb40e36578f-node-exporter-textfile\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.417858 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.417739 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvq7r\" (UniqueName: \"kubernetes.io/projected/0dd584cd-f447-4f04-bc66-efb40e36578f-kube-api-access-pvq7r\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.417858 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.417792 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0dd584cd-f447-4f04-bc66-efb40e36578f-root\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.417858 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.417833 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0dd584cd-f447-4f04-bc66-efb40e36578f-sys\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.418111 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.417869 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/631f4a0a-8dab-49f8-a4da-b493b6d1343a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-l72lr\" (UID: \"631f4a0a-8dab-49f8-a4da-b493b6d1343a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" Jul 21 12:41:15.418111 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.417912 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/631f4a0a-8dab-49f8-a4da-b493b6d1343a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-l72lr\" (UID: \"631f4a0a-8dab-49f8-a4da-b493b6d1343a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" Jul 21 12:41:15.418111 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.417966 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svlrs\" (UniqueName: \"kubernetes.io/projected/631f4a0a-8dab-49f8-a4da-b493b6d1343a-kube-api-access-svlrs\") pod \"openshift-state-metrics-79f6958cdf-l72lr\" (UID: \"631f4a0a-8dab-49f8-a4da-b493b6d1343a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" Jul 21 12:41:15.418111 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.418010 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0dd584cd-f447-4f04-bc66-efb40e36578f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.418111 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.418074 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0dd584cd-f447-4f04-bc66-efb40e36578f-node-exporter-accelerators-collector-config\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.418111 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:41:15.418013 2555 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 21 12:41:15.418111 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.418104 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0dd584cd-f447-4f04-bc66-efb40e36578f-node-exporter-tls\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.418449 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.418144 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0dd584cd-f447-4f04-bc66-efb40e36578f-metrics-client-ca\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.418449 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:41:15.418207 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/631f4a0a-8dab-49f8-a4da-b493b6d1343a-openshift-state-metrics-tls podName:631f4a0a-8dab-49f8-a4da-b493b6d1343a nodeName:}" failed. No retries permitted until 2026-07-21 12:41:15.918187722 +0000 UTC m=+98.536480654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/631f4a0a-8dab-49f8-a4da-b493b6d1343a-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-l72lr" (UID: "631f4a0a-8dab-49f8-a4da-b493b6d1343a") : secret "openshift-state-metrics-tls" not found Jul 21 12:41:15.418449 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.418382 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/631f4a0a-8dab-49f8-a4da-b493b6d1343a-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-l72lr\" (UID: \"631f4a0a-8dab-49f8-a4da-b493b6d1343a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" Jul 21 12:41:15.421058 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.421028 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/631f4a0a-8dab-49f8-a4da-b493b6d1343a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-l72lr\" (UID: \"631f4a0a-8dab-49f8-a4da-b493b6d1343a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" Jul 21 12:41:15.430727 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.430701 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svlrs\" (UniqueName: \"kubernetes.io/projected/631f4a0a-8dab-49f8-a4da-b493b6d1343a-kube-api-access-svlrs\") pod \"openshift-state-metrics-79f6958cdf-l72lr\" (UID: \"631f4a0a-8dab-49f8-a4da-b493b6d1343a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" Jul 21 12:41:15.519269 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.519221 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0dd584cd-f447-4f04-bc66-efb40e36578f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.519464 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.519295 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0dd584cd-f447-4f04-bc66-efb40e36578f-node-exporter-accelerators-collector-config\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.519464 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.519317 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0dd584cd-f447-4f04-bc66-efb40e36578f-node-exporter-tls\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.519464 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.519350 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0dd584cd-f447-4f04-bc66-efb40e36578f-metrics-client-ca\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.519464 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.519380 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0dd584cd-f447-4f04-bc66-efb40e36578f-node-exporter-wtmp\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.519464 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.519403 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0dd584cd-f447-4f04-bc66-efb40e36578f-node-exporter-textfile\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.519464 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.519419 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvq7r\" (UniqueName: \"kubernetes.io/projected/0dd584cd-f447-4f04-bc66-efb40e36578f-kube-api-access-pvq7r\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.519464 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.519438 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0dd584cd-f447-4f04-bc66-efb40e36578f-root\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.519464 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.519466 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0dd584cd-f447-4f04-bc66-efb40e36578f-sys\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.519938 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.519564 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0dd584cd-f447-4f04-bc66-efb40e36578f-sys\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.519999 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.519950 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0dd584cd-f447-4f04-bc66-efb40e36578f-node-exporter-textfile\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.519999 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.519970 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0dd584cd-f447-4f04-bc66-efb40e36578f-node-exporter-wtmp\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.520103 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.520039 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0dd584cd-f447-4f04-bc66-efb40e36578f-node-exporter-accelerators-collector-config\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.520103 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.520093 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0dd584cd-f447-4f04-bc66-efb40e36578f-metrics-client-ca\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.520206 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.520187 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0dd584cd-f447-4f04-bc66-efb40e36578f-root\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.522145 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.522119 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0dd584cd-f447-4f04-bc66-efb40e36578f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.522562 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.522537 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0dd584cd-f447-4f04-bc66-efb40e36578f-node-exporter-tls\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.530472 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.530447 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvq7r\" (UniqueName: \"kubernetes.io/projected/0dd584cd-f447-4f04-bc66-efb40e36578f-kube-api-access-pvq7r\") pod \"node-exporter-d4t5z\" (UID: \"0dd584cd-f447-4f04-bc66-efb40e36578f\") " pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.638629 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.638575 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d4t5z" Jul 21 12:41:15.896893 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:41:15.896864 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dd584cd_f447_4f04_bc66_efb40e36578f.slice/crio-0dc1d2bcc324d271d02ca76ade5e466b0a0457daa56a433ce2448dd99fa501d6 WatchSource:0}: Error finding container 0dc1d2bcc324d271d02ca76ade5e466b0a0457daa56a433ce2448dd99fa501d6: Status 404 returned error can't find the container with id 0dc1d2bcc324d271d02ca76ade5e466b0a0457daa56a433ce2448dd99fa501d6 Jul 21 12:41:15.922850 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.922821 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/631f4a0a-8dab-49f8-a4da-b493b6d1343a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-l72lr\" (UID: \"631f4a0a-8dab-49f8-a4da-b493b6d1343a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" Jul 21 12:41:15.925519 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:15.925489 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/631f4a0a-8dab-49f8-a4da-b493b6d1343a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-l72lr\" (UID: \"631f4a0a-8dab-49f8-a4da-b493b6d1343a\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" Jul 21 12:41:16.217257 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:16.217165 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" Jul 21 12:41:16.231289 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:16.231229 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jblzc" Jul 21 12:41:16.374819 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:16.374790 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr"] Jul 21 12:41:16.379177 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:41:16.379143 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod631f4a0a_8dab_49f8_a4da_b493b6d1343a.slice/crio-9e609ff11397494795812db90dfdf9e660ed5cef3727950c1e7b8e7858406e8d WatchSource:0}: Error finding container 9e609ff11397494795812db90dfdf9e660ed5cef3727950c1e7b8e7858406e8d: Status 404 returned error can't find the container with id 9e609ff11397494795812db90dfdf9e660ed5cef3727950c1e7b8e7858406e8d Jul 21 12:41:16.382730 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:16.382698 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v562r" event={"ID":"037a9656-a505-40ec-b7f8-03b73974213a","Type":"ContainerStarted","Data":"772cb4bb25ea1af3c4324b02c7d323f561db8e50b7a5892ba3271c7b3e2f8bbb"} Jul 21 12:41:16.384189 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:16.384161 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d4t5z" event={"ID":"0dd584cd-f447-4f04-bc66-efb40e36578f","Type":"ContainerStarted","Data":"0dc1d2bcc324d271d02ca76ade5e466b0a0457daa56a433ce2448dd99fa501d6"} Jul 21 12:41:16.402541 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:16.402490 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-v562r" podStartSLOduration=34.27235843 podStartE2EDuration="46.402472369s" totalImagesPullingTime="12.130113939s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:40:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:16.400866571 +0000 UTC m=+99.019159556" watchObservedRunningTime="2026-07-21 12:41:16.402472369 +0000 UTC m=+99.020765316" Jul 21 12:41:17.389640 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.389581 2555 generic.go:356] "Generic (PLEG): container finished" podID="0dd584cd-f447-4f04-bc66-efb40e36578f" containerID="1cf2b966f89154b090c64ce882880c9ac806ab2f846cf9788c29f2d28e960fcf" exitCode=0 Jul 21 12:41:17.390059 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.389671 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d4t5z" event={"ID":"0dd584cd-f447-4f04-bc66-efb40e36578f","Type":"ContainerDied","Data":"1cf2b966f89154b090c64ce882880c9ac806ab2f846cf9788c29f2d28e960fcf"} Jul 21 12:41:17.391631 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.391583 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" event={"ID":"631f4a0a-8dab-49f8-a4da-b493b6d1343a","Type":"ContainerStarted","Data":"06a9b42e484c194f06aa580f5f069a8d0af8624167e3ca2c983828f61587e53e"} Jul 21 12:41:17.391790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.391660 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" event={"ID":"631f4a0a-8dab-49f8-a4da-b493b6d1343a","Type":"ContainerStarted","Data":"975632ac5b7f6295dd12336b059726da5ef0d17100d3756eaa5bdb55a68be8b6"} Jul 21 12:41:17.391790 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.391682 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" event={"ID":"631f4a0a-8dab-49f8-a4da-b493b6d1343a","Type":"ContainerStarted","Data":"9e609ff11397494795812db90dfdf9e660ed5cef3727950c1e7b8e7858406e8d"} Jul 21 12:41:17.435117 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.435086 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7f8d7495db-slrlf"] Jul 21 12:41:17.439381 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.439318 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.441862 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.441821 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 21 12:41:17.441862 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.441838 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 21 12:41:17.442174 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.441821 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 21 12:41:17.442174 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.441969 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 21 12:41:17.442174 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.442109 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-c34rg1buqi305\"" Jul 21 12:41:17.442330 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.442225 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ztt9b\"" Jul 21 12:41:17.442330 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.442299 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 21 12:41:17.452746 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.452722 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7f8d7495db-slrlf"] Jul 21 12:41:17.539063 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.539023 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.539239 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.539074 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.539239 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.539106 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-secret-grpc-tls\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.539239 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.539179 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-secret-thanos-querier-tls\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.539239 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.539224 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfr7v\" (UniqueName: \"kubernetes.io/projected/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-kube-api-access-zfr7v\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.539403 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.539292 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.539403 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.539326 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-metrics-client-ca\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.539403 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.539371 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.640162 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.640124 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.640162 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.640166 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.640422 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.640189 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-secret-grpc-tls\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.640422 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.640208 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-secret-thanos-querier-tls\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.640422 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.640239 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfr7v\" (UniqueName: \"kubernetes.io/projected/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-kube-api-access-zfr7v\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.640422 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.640274 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.640422 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.640303 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-metrics-client-ca\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.640422 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.640354 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.641231 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.641194 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-metrics-client-ca\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.643229 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.643201 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-secret-thanos-querier-tls\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.643229 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.643215 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.643361 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.643274 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.643579 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.643558 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.643676 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.643653 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.643728 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.643714 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-secret-grpc-tls\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.650051 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.650022 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfr7v\" (UniqueName: \"kubernetes.io/projected/b4ad0e99-86fb-4495-9ed2-a977b6b96afc-kube-api-access-zfr7v\") pod \"thanos-querier-7f8d7495db-slrlf\" (UID: \"b4ad0e99-86fb-4495-9ed2-a977b6b96afc\") " pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.750825 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.750716 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:17.895352 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:17.895319 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7f8d7495db-slrlf"] Jul 21 12:41:17.899094 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:41:17.899064 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4ad0e99_86fb_4495_9ed2_a977b6b96afc.slice/crio-f4ac46efe3262c18feda721e7aafa3321a9bca8df9f3f6c485bce41f937b4a23 WatchSource:0}: Error finding container f4ac46efe3262c18feda721e7aafa3321a9bca8df9f3f6c485bce41f937b4a23: Status 404 returned error can't find the container with id f4ac46efe3262c18feda721e7aafa3321a9bca8df9f3f6c485bce41f937b4a23 Jul 21 12:41:18.397150 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:18.397117 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d4t5z" event={"ID":"0dd584cd-f447-4f04-bc66-efb40e36578f","Type":"ContainerStarted","Data":"92bf15230def08c72e7635b62381d590cf76563057259aeeafc674bea57b8c0d"} Jul 21 12:41:18.397537 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:18.397181 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d4t5z" event={"ID":"0dd584cd-f447-4f04-bc66-efb40e36578f","Type":"ContainerStarted","Data":"7577eee03a19ff0301cc96b6690478e8ed26530de3735c2206504d1edf3b786e"} Jul 21 12:41:18.398410 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:18.398376 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" event={"ID":"b4ad0e99-86fb-4495-9ed2-a977b6b96afc","Type":"ContainerStarted","Data":"f4ac46efe3262c18feda721e7aafa3321a9bca8df9f3f6c485bce41f937b4a23"} Jul 21 12:41:18.417079 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:18.417031 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-d4t5z" podStartSLOduration=2.423892892 podStartE2EDuration="3.417012833s" totalImagesPullingTime="918.899449ms" totalInitContainerRuntime="74.220492ms" isStatefulPod=true podCreationTimestamp="2026-07-21 12:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:18.416125527 +0000 UTC m=+101.034418474" watchObservedRunningTime="2026-07-21 12:41:18.417012833 +0000 UTC m=+101.035305780" Jul 21 12:41:19.405424 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:19.405370 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" event={"ID":"631f4a0a-8dab-49f8-a4da-b493b6d1343a","Type":"ContainerStarted","Data":"1a6b581fe317657f8189ac4446a7fa6f219a705e3c65477d8812c526da7e380d"} Jul 21 12:41:19.425202 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:19.425134 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-l72lr" podStartSLOduration=2.587598137 podStartE2EDuration="4.425112014s" totalImagesPullingTime="1.837513877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:19.42505885 +0000 UTC m=+102.043351800" watchObservedRunningTime="2026-07-21 12:41:19.425112014 +0000 UTC m=+102.043404963" Jul 21 12:41:19.920689 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:19.920661 2555 scope.go:117] "RemoveContainer" containerID="97c6110c0f799066aa523f415b46b160a81bd17798b2a547235bf82866add7ab" Jul 21 12:41:20.412167 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:20.412137 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 12:41:20.412622 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:20.412244 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" event={"ID":"af138219-695f-4f5d-8a2f-df2c0a89f117","Type":"ContainerStarted","Data":"fb5200d8508d2e68007bbf23a7e88a5e7dc7ea654148a7cd90386d8989f0058a"} Jul 21 12:41:20.412775 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:20.412759 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:41:20.415058 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:20.415024 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" event={"ID":"b4ad0e99-86fb-4495-9ed2-a977b6b96afc","Type":"ContainerStarted","Data":"0dc38aa6677f94aea9edc7a6afbd6ca3a345ff07d64cb030f183d230d353488e"} Jul 21 12:41:20.415184 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:20.415098 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" event={"ID":"b4ad0e99-86fb-4495-9ed2-a977b6b96afc","Type":"ContainerStarted","Data":"61631a84e60b25cdee284cea7cad6030e12c04aec97540389aa786e753d5cb5d"} Jul 21 12:41:20.415184 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:20.415140 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" event={"ID":"b4ad0e99-86fb-4495-9ed2-a977b6b96afc","Type":"ContainerStarted","Data":"7f1d124d06037532a98575228d0497c0685b6532c8a47e6436dfa21ad1efbb2d"} Jul 21 12:41:20.418978 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:20.418159 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" Jul 21 12:41:20.433491 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:20.433431 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-qnc6t" podStartSLOduration=54.692312861 podStartE2EDuration="58.433410106s" totalImagesPullingTime="3.741097245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:20.433051158 +0000 UTC m=+103.051344110" watchObservedRunningTime="2026-07-21 12:41:20.433410106 +0000 UTC m=+103.051703058" Jul 21 12:41:20.460904 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:20.460867 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-zlwln"] Jul 21 12:41:20.463055 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:20.463034 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-zlwln" Jul 21 12:41:20.465228 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:20.465198 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 21 12:41:20.465358 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:20.465229 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 21 12:41:20.465358 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:20.465229 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-p88g2\"" Jul 21 12:41:20.479430 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:20.479389 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-zlwln"] Jul 21 12:41:20.564563 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:20.564523 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x62tx\" (UniqueName: \"kubernetes.io/projected/73508863-33d6-4f43-b1eb-437b303a601c-kube-api-access-x62tx\") pod \"downloads-5bc67dcd46-zlwln\" (UID: \"73508863-33d6-4f43-b1eb-437b303a601c\") " pod="openshift-console/downloads-5bc67dcd46-zlwln" Jul 21 12:41:20.665557 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:20.665467 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x62tx\" (UniqueName: \"kubernetes.io/projected/73508863-33d6-4f43-b1eb-437b303a601c-kube-api-access-x62tx\") pod \"downloads-5bc67dcd46-zlwln\" (UID: \"73508863-33d6-4f43-b1eb-437b303a601c\") " pod="openshift-console/downloads-5bc67dcd46-zlwln" Jul 21 12:41:20.675758 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:20.675725 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x62tx\" (UniqueName: \"kubernetes.io/projected/73508863-33d6-4f43-b1eb-437b303a601c-kube-api-access-x62tx\") pod \"downloads-5bc67dcd46-zlwln\" (UID: \"73508863-33d6-4f43-b1eb-437b303a601c\") " pod="openshift-console/downloads-5bc67dcd46-zlwln" Jul 21 12:41:20.772761 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:20.772722 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-zlwln" Jul 21 12:41:20.910526 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:20.910496 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-zlwln"] Jul 21 12:41:20.912543 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:41:20.912507 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73508863_33d6_4f43_b1eb_437b303a601c.slice/crio-359b3cb66deeaa210ef1f00f76b802b75bc0367991210438f55a193ebcf42b06 WatchSource:0}: Error finding container 359b3cb66deeaa210ef1f00f76b802b75bc0367991210438f55a193ebcf42b06: Status 404 returned error can't find the container with id 359b3cb66deeaa210ef1f00f76b802b75bc0367991210438f55a193ebcf42b06 Jul 21 12:41:21.421411 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:21.421333 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-zlwln" event={"ID":"73508863-33d6-4f43-b1eb-437b303a601c","Type":"ContainerStarted","Data":"359b3cb66deeaa210ef1f00f76b802b75bc0367991210438f55a193ebcf42b06"} Jul 21 12:41:22.430857 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:22.430820 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" event={"ID":"b4ad0e99-86fb-4495-9ed2-a977b6b96afc","Type":"ContainerStarted","Data":"5f3c5def05936dff96b729f1558a41956760637cddd6b3256d7df5400eb1bdbd"} Jul 21 12:41:22.431321 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:22.430901 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" event={"ID":"b4ad0e99-86fb-4495-9ed2-a977b6b96afc","Type":"ContainerStarted","Data":"cd5640a7fedc7764b73dcee1d9786f9f8860a53fee16b76e00e04c7ebb428d7d"} Jul 21 12:41:22.431321 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:22.430947 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" event={"ID":"b4ad0e99-86fb-4495-9ed2-a977b6b96afc","Type":"ContainerStarted","Data":"fc901df321c1276f4966e19e58fb8dbbe27169af133a3be842846168daffbdbe"} Jul 21 12:41:22.457132 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:22.457071 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" podStartSLOduration=1.625165899 podStartE2EDuration="5.457052021s" totalImagesPullingTime="3.831886122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:41:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:22.455995331 +0000 UTC m=+105.074288278" watchObservedRunningTime="2026-07-21 12:41:22.457052021 +0000 UTC m=+105.075344968" Jul 21 12:41:22.602844 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:22.602808 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7fd754cdd7-bgzfh"] Jul 21 12:41:23.435830 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:23.435805 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:29.447187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:29.447155 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7f8d7495db-slrlf" Jul 21 12:41:35.782421 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:35.782378 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-989cdc8b4-frsxw"] Jul 21 12:41:35.860170 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:35.860122 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-989cdc8b4-frsxw"] Jul 21 12:41:35.860368 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:35.860264 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:35.863399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:35.863047 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 21 12:41:35.863399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:35.863059 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 21 12:41:35.863399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:35.863045 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 21 12:41:35.866249 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:35.866221 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 21 12:41:35.866405 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:35.866258 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 21 12:41:35.866405 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:35.866293 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9m7pl\"" Jul 21 12:41:35.870805 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:35.870780 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 21 12:41:35.913922 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:35.913710 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-service-ca\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:35.913922 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:35.913828 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-oauth-serving-cert\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:35.913922 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:35.913885 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-console-config\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:35.914206 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:35.913965 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d123a458-415c-429a-bc88-124354baeed7-console-serving-cert\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:35.914206 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:35.914001 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d123a458-415c-429a-bc88-124354baeed7-console-oauth-config\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:35.914206 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:35.914031 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqjph\" (UniqueName: \"kubernetes.io/projected/d123a458-415c-429a-bc88-124354baeed7-kube-api-access-tqjph\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:35.914206 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:35.914055 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-trusted-ca-bundle\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:36.015022 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:36.014981 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-oauth-serving-cert\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:36.015207 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:36.015067 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-console-config\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:36.015281 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:36.015205 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d123a458-415c-429a-bc88-124354baeed7-console-serving-cert\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:36.015281 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:36.015234 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d123a458-415c-429a-bc88-124354baeed7-console-oauth-config\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:36.015281 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:36.015266 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqjph\" (UniqueName: \"kubernetes.io/projected/d123a458-415c-429a-bc88-124354baeed7-kube-api-access-tqjph\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:36.015416 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:36.015294 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-trusted-ca-bundle\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:36.015416 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:36.015365 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-service-ca\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:36.015947 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:36.015894 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-console-config\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:36.015947 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:36.015923 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-oauth-serving-cert\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:36.016137 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:36.016046 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-service-ca\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:36.016807 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:36.016781 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-trusted-ca-bundle\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:36.018236 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:36.018192 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d123a458-415c-429a-bc88-124354baeed7-console-serving-cert\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:36.018342 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:36.018296 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d123a458-415c-429a-bc88-124354baeed7-console-oauth-config\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:36.026443 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:36.026380 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqjph\" (UniqueName: \"kubernetes.io/projected/d123a458-415c-429a-bc88-124354baeed7-kube-api-access-tqjph\") pod \"console-989cdc8b4-frsxw\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:36.173152 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:36.173104 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:38.850638 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:38.850585 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-989cdc8b4-frsxw"] Jul 21 12:41:38.853511 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:41:38.853477 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd123a458_415c_429a_bc88_124354baeed7.slice/crio-f3f4da684cb1581a25870bcc864f4253ccecb5911bb15ba74eca012265cf107a WatchSource:0}: Error finding container f3f4da684cb1581a25870bcc864f4253ccecb5911bb15ba74eca012265cf107a: Status 404 returned error can't find the container with id f3f4da684cb1581a25870bcc864f4253ccecb5911bb15ba74eca012265cf107a Jul 21 12:41:39.514557 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:39.514510 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-zlwln" event={"ID":"73508863-33d6-4f43-b1eb-437b303a601c","Type":"ContainerStarted","Data":"a3c1a8b733ee97e754f54ae34aab0f54507e08f0ad5fe30c139f0abae7e8ec64"} Jul 21 12:41:39.516044 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:39.515884 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-zlwln" Jul 21 12:41:39.518119 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:39.518089 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-989cdc8b4-frsxw" event={"ID":"d123a458-415c-429a-bc88-124354baeed7","Type":"ContainerStarted","Data":"f3f4da684cb1581a25870bcc864f4253ccecb5911bb15ba74eca012265cf107a"} Jul 21 12:41:39.527287 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:39.527250 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-zlwln" Jul 21 12:41:39.561419 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:39.561351 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-zlwln" podStartSLOduration=1.571220227 podStartE2EDuration="19.561328884s" totalImagesPullingTime="17.990108657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:41:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:39.540091701 +0000 UTC m=+122.158384650" watchObservedRunningTime="2026-07-21 12:41:39.561328884 +0000 UTC m=+122.179621833" Jul 21 12:41:43.557674 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:43.557625 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-989cdc8b4-frsxw" event={"ID":"d123a458-415c-429a-bc88-124354baeed7","Type":"ContainerStarted","Data":"d62e18e9a74c8bbf8c6bde2d226787f6a0586fcc2764171890d33bf4c7cdfc7f"} Jul 21 12:41:43.580740 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:43.580666 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-989cdc8b4-frsxw" podStartSLOduration=4.577950561 podStartE2EDuration="8.580643744s" totalImagesPullingTime="4.002693183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:41:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:43.579435853 +0000 UTC m=+126.197728805" watchObservedRunningTime="2026-07-21 12:41:43.580643744 +0000 UTC m=+126.198936692" Jul 21 12:41:46.173804 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:46.173766 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:46.174308 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:46.174098 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:46.179389 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:46.179366 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:46.576158 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:46.576075 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:41:47.627005 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:47.626958 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" podUID="b6e594b6-715f-4157-b411-94fbf7a3bdce" containerName="registry" containerID="cri-o://803052ce486ada3570a56ea40e8b7b4b29a982a918c6798d4491ac815fb7e14f" gracePeriod=30 Jul 21 12:41:47.730184 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:47.729800 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs\") pod \"network-metrics-daemon-z65kc\" (UID: \"eb0efdf9-1311-4257-9f40-3c35bee29d66\") " pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:41:47.732751 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:47.732721 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb0efdf9-1311-4257-9f40-3c35bee29d66-metrics-certs\") pod \"network-metrics-daemon-z65kc\" (UID: \"eb0efdf9-1311-4257-9f40-3c35bee29d66\") " pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:41:47.938636 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:47.938583 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tqbcg\"" Jul 21 12:41:47.946909 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:47.946877 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z65kc" Jul 21 12:41:47.953991 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:47.953958 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:41:48.032220 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.032188 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6e594b6-715f-4157-b411-94fbf7a3bdce-installation-pull-secrets\") pod \"b6e594b6-715f-4157-b411-94fbf7a3bdce\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " Jul 21 12:41:48.032390 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.032244 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6e594b6-715f-4157-b411-94fbf7a3bdce-trusted-ca\") pod \"b6e594b6-715f-4157-b411-94fbf7a3bdce\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " Jul 21 12:41:48.032390 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.032303 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6e594b6-715f-4157-b411-94fbf7a3bdce-image-registry-private-configuration\") pod \"b6e594b6-715f-4157-b411-94fbf7a3bdce\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " Jul 21 12:41:48.032390 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.032354 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ns445\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-kube-api-access-ns445\") pod \"b6e594b6-715f-4157-b411-94fbf7a3bdce\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " Jul 21 12:41:48.032390 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.032383 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-bound-sa-token\") pod \"b6e594b6-715f-4157-b411-94fbf7a3bdce\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " Jul 21 12:41:48.032616 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.032407 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-certificates\") pod \"b6e594b6-715f-4157-b411-94fbf7a3bdce\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " Jul 21 12:41:48.032616 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.032436 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls\") pod \"b6e594b6-715f-4157-b411-94fbf7a3bdce\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " Jul 21 12:41:48.032616 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.032469 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6e594b6-715f-4157-b411-94fbf7a3bdce-ca-trust-extracted\") pod \"b6e594b6-715f-4157-b411-94fbf7a3bdce\" (UID: \"b6e594b6-715f-4157-b411-94fbf7a3bdce\") " Jul 21 12:41:48.032989 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.032956 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b6e594b6-715f-4157-b411-94fbf7a3bdce" (UID: "b6e594b6-715f-4157-b411-94fbf7a3bdce"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:41:48.033110 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.033088 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6e594b6-715f-4157-b411-94fbf7a3bdce-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b6e594b6-715f-4157-b411-94fbf7a3bdce" (UID: "b6e594b6-715f-4157-b411-94fbf7a3bdce"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:41:48.037717 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.037627 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b6e594b6-715f-4157-b411-94fbf7a3bdce" (UID: "b6e594b6-715f-4157-b411-94fbf7a3bdce"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:41:48.041898 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.039498 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b6e594b6-715f-4157-b411-94fbf7a3bdce" (UID: "b6e594b6-715f-4157-b411-94fbf7a3bdce"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:41:48.046064 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.046034 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-kube-api-access-ns445" (OuterVolumeSpecName: "kube-api-access-ns445") pod "b6e594b6-715f-4157-b411-94fbf7a3bdce" (UID: "b6e594b6-715f-4157-b411-94fbf7a3bdce"). InnerVolumeSpecName "kube-api-access-ns445". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:41:48.047010 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.046951 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6e594b6-715f-4157-b411-94fbf7a3bdce-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b6e594b6-715f-4157-b411-94fbf7a3bdce" (UID: "b6e594b6-715f-4157-b411-94fbf7a3bdce"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:41:48.048041 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.048000 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6e594b6-715f-4157-b411-94fbf7a3bdce-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b6e594b6-715f-4157-b411-94fbf7a3bdce" (UID: "b6e594b6-715f-4157-b411-94fbf7a3bdce"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:41:48.050449 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.050399 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6e594b6-715f-4157-b411-94fbf7a3bdce-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b6e594b6-715f-4157-b411-94fbf7a3bdce" (UID: "b6e594b6-715f-4157-b411-94fbf7a3bdce"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:41:48.133362 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.133168 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ns445\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-kube-api-access-ns445\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:41:48.133362 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.133204 2555 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-bound-sa-token\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:41:48.133362 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.133219 2555 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-certificates\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:41:48.133362 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.133236 2555 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b6e594b6-715f-4157-b411-94fbf7a3bdce-registry-tls\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:41:48.133362 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.133251 2555 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b6e594b6-715f-4157-b411-94fbf7a3bdce-ca-trust-extracted\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:41:48.133362 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.133283 2555 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b6e594b6-715f-4157-b411-94fbf7a3bdce-installation-pull-secrets\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:41:48.133362 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.133297 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b6e594b6-715f-4157-b411-94fbf7a3bdce-trusted-ca\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:41:48.133362 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.133311 2555 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b6e594b6-715f-4157-b411-94fbf7a3bdce-image-registry-private-configuration\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:41:48.187019 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.186908 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-z65kc"] Jul 21 12:41:48.190622 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:41:48.190563 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb0efdf9_1311_4257_9f40_3c35bee29d66.slice/crio-2c55eaeaf5af7b0d44ca9c1bd1fdba14a20e2f0ab6e63cc256b8c5a10db20771 WatchSource:0}: Error finding container 2c55eaeaf5af7b0d44ca9c1bd1fdba14a20e2f0ab6e63cc256b8c5a10db20771: Status 404 returned error can't find the container with id 2c55eaeaf5af7b0d44ca9c1bd1fdba14a20e2f0ab6e63cc256b8c5a10db20771 Jul 21 12:41:48.580150 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.580061 2555 generic.go:356] "Generic (PLEG): container finished" podID="b6e594b6-715f-4157-b411-94fbf7a3bdce" containerID="803052ce486ada3570a56ea40e8b7b4b29a982a918c6798d4491ac815fb7e14f" exitCode=0 Jul 21 12:41:48.580150 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.580111 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" Jul 21 12:41:48.580492 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.580161 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" event={"ID":"b6e594b6-715f-4157-b411-94fbf7a3bdce","Type":"ContainerDied","Data":"803052ce486ada3570a56ea40e8b7b4b29a982a918c6798d4491ac815fb7e14f"} Jul 21 12:41:48.580492 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.580220 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fd754cdd7-bgzfh" event={"ID":"b6e594b6-715f-4157-b411-94fbf7a3bdce","Type":"ContainerDied","Data":"9faca21b86d1e62bf1b23cad6e88e47d4b3591f4d8acc664d88ed9f8fcc6d7d1"} Jul 21 12:41:48.580492 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.580250 2555 scope.go:117] "RemoveContainer" containerID="803052ce486ada3570a56ea40e8b7b4b29a982a918c6798d4491ac815fb7e14f" Jul 21 12:41:48.581427 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.581402 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z65kc" event={"ID":"eb0efdf9-1311-4257-9f40-3c35bee29d66","Type":"ContainerStarted","Data":"2c55eaeaf5af7b0d44ca9c1bd1fdba14a20e2f0ab6e63cc256b8c5a10db20771"} Jul 21 12:41:48.589085 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.589065 2555 scope.go:117] "RemoveContainer" containerID="803052ce486ada3570a56ea40e8b7b4b29a982a918c6798d4491ac815fb7e14f" Jul 21 12:41:48.589391 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:41:48.589370 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"803052ce486ada3570a56ea40e8b7b4b29a982a918c6798d4491ac815fb7e14f\": container with ID starting with 803052ce486ada3570a56ea40e8b7b4b29a982a918c6798d4491ac815fb7e14f not found: ID does not exist" containerID="803052ce486ada3570a56ea40e8b7b4b29a982a918c6798d4491ac815fb7e14f" Jul 21 12:41:48.589460 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.589403 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"803052ce486ada3570a56ea40e8b7b4b29a982a918c6798d4491ac815fb7e14f"} err="failed to get container status \"803052ce486ada3570a56ea40e8b7b4b29a982a918c6798d4491ac815fb7e14f\": rpc error: code = NotFound desc = could not find container \"803052ce486ada3570a56ea40e8b7b4b29a982a918c6798d4491ac815fb7e14f\": container with ID starting with 803052ce486ada3570a56ea40e8b7b4b29a982a918c6798d4491ac815fb7e14f not found: ID does not exist" Jul 21 12:41:48.605087 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.605049 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7fd754cdd7-bgzfh"] Jul 21 12:41:48.613305 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:48.613279 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7fd754cdd7-bgzfh"] Jul 21 12:41:49.925187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:49.925152 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6e594b6-715f-4157-b411-94fbf7a3bdce" path="/var/lib/kubelet/pods/b6e594b6-715f-4157-b411-94fbf7a3bdce/volumes" Jul 21 12:41:50.591664 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:50.591628 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z65kc" event={"ID":"eb0efdf9-1311-4257-9f40-3c35bee29d66","Type":"ContainerStarted","Data":"00a2483e7784db4da10620c81ffe4749130a96e9939e7a6c909141d3974bf6a2"} Jul 21 12:41:50.591840 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:50.591678 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z65kc" event={"ID":"eb0efdf9-1311-4257-9f40-3c35bee29d66","Type":"ContainerStarted","Data":"35bb1c2b96de56c7d2cb9df8c863fb0815ea9a42e47d63b5eec1d823f6e1aea0"} Jul 21 12:41:50.610308 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:41:50.610255 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-z65kc" podStartSLOduration=132.209618939 podStartE2EDuration="2m13.61023655s" totalImagesPullingTime="1.400617611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:39:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:50.608418164 +0000 UTC m=+133.226711111" watchObservedRunningTime="2026-07-21 12:41:50.61023655 +0000 UTC m=+133.228529498" Jul 21 12:42:43.835034 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:43.834193 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b45c597bd-7kwn5"] Jul 21 12:42:43.835034 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:43.834681 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6e594b6-715f-4157-b411-94fbf7a3bdce" containerName="registry" Jul 21 12:42:43.835034 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:43.834698 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6e594b6-715f-4157-b411-94fbf7a3bdce" containerName="registry" Jul 21 12:42:43.835034 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:43.834770 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6e594b6-715f-4157-b411-94fbf7a3bdce" containerName="registry" Jul 21 12:42:43.837997 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:43.837417 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:43.850945 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:43.850897 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b45c597bd-7kwn5"] Jul 21 12:42:43.991257 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:43.991211 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-trusted-ca-bundle\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:43.991257 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:43.991259 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-service-ca\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:43.991496 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:43.991345 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-console-config\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:43.991496 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:43.991389 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-oauth-serving-cert\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:43.991496 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:43.991438 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7ff3558-d86f-4e72-9e92-43e2c2354666-console-serving-cert\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:43.991496 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:43.991462 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7ff3558-d86f-4e72-9e92-43e2c2354666-console-oauth-config\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:43.991496 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:43.991478 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56c5r\" (UniqueName: \"kubernetes.io/projected/f7ff3558-d86f-4e72-9e92-43e2c2354666-kube-api-access-56c5r\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:44.092637 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:44.092579 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-trusted-ca-bundle\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:44.093735 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:44.093703 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-service-ca\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:44.093882 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:44.093840 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-console-config\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:44.093882 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:44.093874 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-oauth-serving-cert\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:44.093996 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:44.093886 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-trusted-ca-bundle\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:44.094056 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:44.093995 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7ff3558-d86f-4e72-9e92-43e2c2354666-console-serving-cert\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:44.094056 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:44.094029 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7ff3558-d86f-4e72-9e92-43e2c2354666-console-oauth-config\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:44.094146 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:44.094055 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56c5r\" (UniqueName: \"kubernetes.io/projected/f7ff3558-d86f-4e72-9e92-43e2c2354666-kube-api-access-56c5r\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:44.094563 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:44.094535 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-oauth-serving-cert\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:44.094709 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:44.094535 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-console-config\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:44.095040 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:44.094995 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-service-ca\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:44.096901 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:44.096877 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7ff3558-d86f-4e72-9e92-43e2c2354666-console-serving-cert\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:44.097008 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:44.096979 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7ff3558-d86f-4e72-9e92-43e2c2354666-console-oauth-config\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:44.103073 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:44.103048 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56c5r\" (UniqueName: \"kubernetes.io/projected/f7ff3558-d86f-4e72-9e92-43e2c2354666-kube-api-access-56c5r\") pod \"console-6b45c597bd-7kwn5\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:44.152273 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:44.152238 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:44.313979 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:44.313941 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b45c597bd-7kwn5"] Jul 21 12:42:44.799177 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:44.799082 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b45c597bd-7kwn5" event={"ID":"f7ff3558-d86f-4e72-9e92-43e2c2354666","Type":"ContainerStarted","Data":"e850f87ff9ad9119e79b15c6bdd4b2ba9b9c183177391227419aad46cd9da1d9"} Jul 21 12:42:44.799177 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:44.799127 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b45c597bd-7kwn5" event={"ID":"f7ff3558-d86f-4e72-9e92-43e2c2354666","Type":"ContainerStarted","Data":"8a35a19aaa5822341da98bc9e9e299876c6d32eaa1f7e2c3a17fc7fa91686726"} Jul 21 12:42:44.821520 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:44.821469 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b45c597bd-7kwn5" podStartSLOduration=1.821452522 podStartE2EDuration="1.821452522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:42:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:42:44.819762417 +0000 UTC m=+187.438055365" watchObservedRunningTime="2026-07-21 12:42:44.821452522 +0000 UTC m=+187.439745469" Jul 21 12:42:54.152653 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:54.152616 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:54.153176 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:54.152693 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:54.157614 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:54.157576 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:54.839964 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:54.839933 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:42:54.891324 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:42:54.891292 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-989cdc8b4-frsxw"] Jul 21 12:43:19.918055 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:19.918008 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-989cdc8b4-frsxw" podUID="d123a458-415c-429a-bc88-124354baeed7" containerName="console" containerID="cri-o://d62e18e9a74c8bbf8c6bde2d226787f6a0586fcc2764171890d33bf4c7cdfc7f" gracePeriod=15 Jul 21 12:43:20.167266 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.167240 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:43:20.254436 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.254337 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-service-ca\") pod \"d123a458-415c-429a-bc88-124354baeed7\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " Jul 21 12:43:20.254436 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.254405 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-console-config\") pod \"d123a458-415c-429a-bc88-124354baeed7\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " Jul 21 12:43:20.254436 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.254435 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-oauth-serving-cert\") pod \"d123a458-415c-429a-bc88-124354baeed7\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " Jul 21 12:43:20.254739 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.254553 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqjph\" (UniqueName: \"kubernetes.io/projected/d123a458-415c-429a-bc88-124354baeed7-kube-api-access-tqjph\") pod \"d123a458-415c-429a-bc88-124354baeed7\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " Jul 21 12:43:20.254739 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.254644 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d123a458-415c-429a-bc88-124354baeed7-console-oauth-config\") pod \"d123a458-415c-429a-bc88-124354baeed7\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " Jul 21 12:43:20.254739 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.254676 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d123a458-415c-429a-bc88-124354baeed7-console-serving-cert\") pod \"d123a458-415c-429a-bc88-124354baeed7\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " Jul 21 12:43:20.254739 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.254704 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-trusted-ca-bundle\") pod \"d123a458-415c-429a-bc88-124354baeed7\" (UID: \"d123a458-415c-429a-bc88-124354baeed7\") " Jul 21 12:43:20.254935 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.254778 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-console-config" (OuterVolumeSpecName: "console-config") pod "d123a458-415c-429a-bc88-124354baeed7" (UID: "d123a458-415c-429a-bc88-124354baeed7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:43:20.254935 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.254818 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-service-ca" (OuterVolumeSpecName: "service-ca") pod "d123a458-415c-429a-bc88-124354baeed7" (UID: "d123a458-415c-429a-bc88-124354baeed7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:43:20.254935 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.254888 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d123a458-415c-429a-bc88-124354baeed7" (UID: "d123a458-415c-429a-bc88-124354baeed7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:43:20.255076 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.254977 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-service-ca\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:43:20.255076 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.254994 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-console-config\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:43:20.255076 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.255008 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-oauth-serving-cert\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:43:20.255230 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.255161 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d123a458-415c-429a-bc88-124354baeed7" (UID: "d123a458-415c-429a-bc88-124354baeed7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:43:20.257013 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.256988 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d123a458-415c-429a-bc88-124354baeed7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d123a458-415c-429a-bc88-124354baeed7" (UID: "d123a458-415c-429a-bc88-124354baeed7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:43:20.257088 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.257075 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d123a458-415c-429a-bc88-124354baeed7-kube-api-access-tqjph" (OuterVolumeSpecName: "kube-api-access-tqjph") pod "d123a458-415c-429a-bc88-124354baeed7" (UID: "d123a458-415c-429a-bc88-124354baeed7"). InnerVolumeSpecName "kube-api-access-tqjph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:43:20.257131 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.257074 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d123a458-415c-429a-bc88-124354baeed7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d123a458-415c-429a-bc88-124354baeed7" (UID: "d123a458-415c-429a-bc88-124354baeed7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:43:20.356102 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.356062 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqjph\" (UniqueName: \"kubernetes.io/projected/d123a458-415c-429a-bc88-124354baeed7-kube-api-access-tqjph\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:43:20.356102 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.356092 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d123a458-415c-429a-bc88-124354baeed7-console-oauth-config\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:43:20.356102 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.356102 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d123a458-415c-429a-bc88-124354baeed7-console-serving-cert\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:43:20.356102 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.356110 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d123a458-415c-429a-bc88-124354baeed7-trusted-ca-bundle\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:43:20.939680 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.939628 2555 generic.go:356] "Generic (PLEG): container finished" podID="d123a458-415c-429a-bc88-124354baeed7" containerID="d62e18e9a74c8bbf8c6bde2d226787f6a0586fcc2764171890d33bf4c7cdfc7f" exitCode=0 Jul 21 12:43:20.939680 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.939676 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-989cdc8b4-frsxw" Jul 21 12:43:20.940166 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.939702 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-989cdc8b4-frsxw" event={"ID":"d123a458-415c-429a-bc88-124354baeed7","Type":"ContainerDied","Data":"d62e18e9a74c8bbf8c6bde2d226787f6a0586fcc2764171890d33bf4c7cdfc7f"} Jul 21 12:43:20.940166 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.939766 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-989cdc8b4-frsxw" event={"ID":"d123a458-415c-429a-bc88-124354baeed7","Type":"ContainerDied","Data":"f3f4da684cb1581a25870bcc864f4253ccecb5911bb15ba74eca012265cf107a"} Jul 21 12:43:20.940166 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.939794 2555 scope.go:117] "RemoveContainer" containerID="d62e18e9a74c8bbf8c6bde2d226787f6a0586fcc2764171890d33bf4c7cdfc7f" Jul 21 12:43:20.948979 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.948958 2555 scope.go:117] "RemoveContainer" containerID="d62e18e9a74c8bbf8c6bde2d226787f6a0586fcc2764171890d33bf4c7cdfc7f" Jul 21 12:43:20.949275 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:43:20.949257 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d62e18e9a74c8bbf8c6bde2d226787f6a0586fcc2764171890d33bf4c7cdfc7f\": container with ID starting with d62e18e9a74c8bbf8c6bde2d226787f6a0586fcc2764171890d33bf4c7cdfc7f not found: ID does not exist" containerID="d62e18e9a74c8bbf8c6bde2d226787f6a0586fcc2764171890d33bf4c7cdfc7f" Jul 21 12:43:20.949343 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.949280 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d62e18e9a74c8bbf8c6bde2d226787f6a0586fcc2764171890d33bf4c7cdfc7f"} err="failed to get container status \"d62e18e9a74c8bbf8c6bde2d226787f6a0586fcc2764171890d33bf4c7cdfc7f\": rpc error: code = NotFound desc = could not find container \"d62e18e9a74c8bbf8c6bde2d226787f6a0586fcc2764171890d33bf4c7cdfc7f\": container with ID starting with d62e18e9a74c8bbf8c6bde2d226787f6a0586fcc2764171890d33bf4c7cdfc7f not found: ID does not exist" Jul 21 12:43:20.961788 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.961758 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-989cdc8b4-frsxw"] Jul 21 12:43:20.966671 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:20.966636 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-989cdc8b4-frsxw"] Jul 21 12:43:21.924555 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:21.924522 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d123a458-415c-429a-bc88-124354baeed7" path="/var/lib/kubelet/pods/d123a458-415c-429a-bc88-124354baeed7/volumes" Jul 21 12:43:51.491334 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.491298 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66c4d7cd5b-g46h4"] Jul 21 12:43:51.491801 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.491667 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d123a458-415c-429a-bc88-124354baeed7" containerName="console" Jul 21 12:43:51.491801 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.491680 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d123a458-415c-429a-bc88-124354baeed7" containerName="console" Jul 21 12:43:51.491801 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.491737 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d123a458-415c-429a-bc88-124354baeed7" containerName="console" Jul 21 12:43:51.494660 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.494630 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.505336 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.505308 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66c4d7cd5b-g46h4"] Jul 21 12:43:51.600066 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.600022 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9wqb\" (UniqueName: \"kubernetes.io/projected/f7b03598-eacd-442b-b66d-88497e54683a-kube-api-access-r9wqb\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.600066 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.600067 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7b03598-eacd-442b-b66d-88497e54683a-console-serving-cert\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.600313 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.600091 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7b03598-eacd-442b-b66d-88497e54683a-console-oauth-config\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.600313 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.600157 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-console-config\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.600313 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.600224 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-trusted-ca-bundle\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.600313 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.600301 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-oauth-serving-cert\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.600456 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.600324 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-service-ca\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.701148 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.701097 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-oauth-serving-cert\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.701148 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.701149 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-service-ca\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.701407 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.701179 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9wqb\" (UniqueName: \"kubernetes.io/projected/f7b03598-eacd-442b-b66d-88497e54683a-kube-api-access-r9wqb\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.701407 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.701202 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7b03598-eacd-442b-b66d-88497e54683a-console-serving-cert\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.701407 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.701223 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7b03598-eacd-442b-b66d-88497e54683a-console-oauth-config\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.701407 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.701253 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-console-config\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.701407 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.701303 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-trusted-ca-bundle\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.702036 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.702010 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-service-ca\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.702143 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.701974 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-oauth-serving-cert\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.702206 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.702010 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-console-config\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.702206 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.702191 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-trusted-ca-bundle\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.703830 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.703808 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7b03598-eacd-442b-b66d-88497e54683a-console-oauth-config\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.703932 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.703915 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7b03598-eacd-442b-b66d-88497e54683a-console-serving-cert\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.713999 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.713972 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9wqb\" (UniqueName: \"kubernetes.io/projected/f7b03598-eacd-442b-b66d-88497e54683a-kube-api-access-r9wqb\") pod \"console-66c4d7cd5b-g46h4\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.805114 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.805019 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:43:51.945264 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:51.945231 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66c4d7cd5b-g46h4"] Jul 21 12:43:51.948431 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:43:51.948403 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7b03598_eacd_442b_b66d_88497e54683a.slice/crio-69fc9d81791a75ace13f493a2cd5349dd54d0e71ee2fe19663ccb4031a9bb5bd WatchSource:0}: Error finding container 69fc9d81791a75ace13f493a2cd5349dd54d0e71ee2fe19663ccb4031a9bb5bd: Status 404 returned error can't find the container with id 69fc9d81791a75ace13f493a2cd5349dd54d0e71ee2fe19663ccb4031a9bb5bd Jul 21 12:43:52.057195 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:52.057105 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66c4d7cd5b-g46h4" event={"ID":"f7b03598-eacd-442b-b66d-88497e54683a","Type":"ContainerStarted","Data":"a4af9ed236ade3960e829b730117e58845a70a5dde46ebe7a93a877a154e02af"} Jul 21 12:43:52.057195 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:52.057161 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66c4d7cd5b-g46h4" event={"ID":"f7b03598-eacd-442b-b66d-88497e54683a","Type":"ContainerStarted","Data":"69fc9d81791a75ace13f493a2cd5349dd54d0e71ee2fe19663ccb4031a9bb5bd"} Jul 21 12:43:52.076728 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:43:52.076679 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66c4d7cd5b-g46h4" podStartSLOduration=1.076661132 podStartE2EDuration="1.076661132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:43:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:43:52.075680022 +0000 UTC m=+254.693972970" watchObservedRunningTime="2026-07-21 12:43:52.076661132 +0000 UTC m=+254.694954079" Jul 21 12:44:01.805618 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:01.805497 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:44:01.805618 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:01.805578 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:44:01.810640 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:01.810614 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:44:02.097847 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:02.097810 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:44:02.168560 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:02.168527 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b45c597bd-7kwn5"] Jul 21 12:44:27.197688 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.197629 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b45c597bd-7kwn5" podUID="f7ff3558-d86f-4e72-9e92-43e2c2354666" containerName="console" containerID="cri-o://e850f87ff9ad9119e79b15c6bdd4b2ba9b9c183177391227419aad46cd9da1d9" gracePeriod=15 Jul 21 12:44:27.444943 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.444912 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:44:27.481662 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.481560 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-trusted-ca-bundle\") pod \"f7ff3558-d86f-4e72-9e92-43e2c2354666\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " Jul 21 12:44:27.481662 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.481615 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7ff3558-d86f-4e72-9e92-43e2c2354666-console-oauth-config\") pod \"f7ff3558-d86f-4e72-9e92-43e2c2354666\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " Jul 21 12:44:27.481662 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.481636 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7ff3558-d86f-4e72-9e92-43e2c2354666-console-serving-cert\") pod \"f7ff3558-d86f-4e72-9e92-43e2c2354666\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " Jul 21 12:44:27.481939 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.481686 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-service-ca\") pod \"f7ff3558-d86f-4e72-9e92-43e2c2354666\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " Jul 21 12:44:27.481939 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.481713 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-oauth-serving-cert\") pod \"f7ff3558-d86f-4e72-9e92-43e2c2354666\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " Jul 21 12:44:27.481939 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.481766 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56c5r\" (UniqueName: \"kubernetes.io/projected/f7ff3558-d86f-4e72-9e92-43e2c2354666-kube-api-access-56c5r\") pod \"f7ff3558-d86f-4e72-9e92-43e2c2354666\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " Jul 21 12:44:27.481939 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.481814 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-console-config\") pod \"f7ff3558-d86f-4e72-9e92-43e2c2354666\" (UID: \"f7ff3558-d86f-4e72-9e92-43e2c2354666\") " Jul 21 12:44:27.482128 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.482050 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f7ff3558-d86f-4e72-9e92-43e2c2354666" (UID: "f7ff3558-d86f-4e72-9e92-43e2c2354666"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:44:27.483686 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.482231 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f7ff3558-d86f-4e72-9e92-43e2c2354666" (UID: "f7ff3558-d86f-4e72-9e92-43e2c2354666"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:44:27.483686 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.482245 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-console-config" (OuterVolumeSpecName: "console-config") pod "f7ff3558-d86f-4e72-9e92-43e2c2354666" (UID: "f7ff3558-d86f-4e72-9e92-43e2c2354666"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:44:27.483686 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.482480 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-service-ca" (OuterVolumeSpecName: "service-ca") pod "f7ff3558-d86f-4e72-9e92-43e2c2354666" (UID: "f7ff3558-d86f-4e72-9e92-43e2c2354666"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:44:27.484198 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.484165 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7ff3558-d86f-4e72-9e92-43e2c2354666-kube-api-access-56c5r" (OuterVolumeSpecName: "kube-api-access-56c5r") pod "f7ff3558-d86f-4e72-9e92-43e2c2354666" (UID: "f7ff3558-d86f-4e72-9e92-43e2c2354666"). InnerVolumeSpecName "kube-api-access-56c5r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:44:27.484425 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.484408 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7ff3558-d86f-4e72-9e92-43e2c2354666-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f7ff3558-d86f-4e72-9e92-43e2c2354666" (UID: "f7ff3558-d86f-4e72-9e92-43e2c2354666"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:44:27.484512 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.484498 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7ff3558-d86f-4e72-9e92-43e2c2354666-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f7ff3558-d86f-4e72-9e92-43e2c2354666" (UID: "f7ff3558-d86f-4e72-9e92-43e2c2354666"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:44:27.582753 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.582714 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-service-ca\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:44:27.582753 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.582747 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-oauth-serving-cert\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:44:27.582753 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.582757 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56c5r\" (UniqueName: \"kubernetes.io/projected/f7ff3558-d86f-4e72-9e92-43e2c2354666-kube-api-access-56c5r\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:44:27.582985 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.582766 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-console-config\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:44:27.582985 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.582776 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7ff3558-d86f-4e72-9e92-43e2c2354666-trusted-ca-bundle\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:44:27.582985 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.582784 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7ff3558-d86f-4e72-9e92-43e2c2354666-console-oauth-config\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:44:27.582985 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:27.582793 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7ff3558-d86f-4e72-9e92-43e2c2354666-console-serving-cert\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:44:28.195368 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:28.195274 2555 generic.go:356] "Generic (PLEG): container finished" podID="f7ff3558-d86f-4e72-9e92-43e2c2354666" containerID="e850f87ff9ad9119e79b15c6bdd4b2ba9b9c183177391227419aad46cd9da1d9" exitCode=0 Jul 21 12:44:28.195368 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:28.195326 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b45c597bd-7kwn5" Jul 21 12:44:28.195624 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:28.195332 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b45c597bd-7kwn5" event={"ID":"f7ff3558-d86f-4e72-9e92-43e2c2354666","Type":"ContainerDied","Data":"e850f87ff9ad9119e79b15c6bdd4b2ba9b9c183177391227419aad46cd9da1d9"} Jul 21 12:44:28.195624 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:28.195467 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b45c597bd-7kwn5" event={"ID":"f7ff3558-d86f-4e72-9e92-43e2c2354666","Type":"ContainerDied","Data":"8a35a19aaa5822341da98bc9e9e299876c6d32eaa1f7e2c3a17fc7fa91686726"} Jul 21 12:44:28.195624 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:28.195495 2555 scope.go:117] "RemoveContainer" containerID="e850f87ff9ad9119e79b15c6bdd4b2ba9b9c183177391227419aad46cd9da1d9" Jul 21 12:44:28.203885 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:28.203687 2555 scope.go:117] "RemoveContainer" containerID="e850f87ff9ad9119e79b15c6bdd4b2ba9b9c183177391227419aad46cd9da1d9" Jul 21 12:44:28.204090 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:44:28.203987 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e850f87ff9ad9119e79b15c6bdd4b2ba9b9c183177391227419aad46cd9da1d9\": container with ID starting with e850f87ff9ad9119e79b15c6bdd4b2ba9b9c183177391227419aad46cd9da1d9 not found: ID does not exist" containerID="e850f87ff9ad9119e79b15c6bdd4b2ba9b9c183177391227419aad46cd9da1d9" Jul 21 12:44:28.204090 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:28.204013 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e850f87ff9ad9119e79b15c6bdd4b2ba9b9c183177391227419aad46cd9da1d9"} err="failed to get container status \"e850f87ff9ad9119e79b15c6bdd4b2ba9b9c183177391227419aad46cd9da1d9\": rpc error: code = NotFound desc = could not find container \"e850f87ff9ad9119e79b15c6bdd4b2ba9b9c183177391227419aad46cd9da1d9\": container with ID starting with e850f87ff9ad9119e79b15c6bdd4b2ba9b9c183177391227419aad46cd9da1d9 not found: ID does not exist" Jul 21 12:44:28.219443 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:28.219401 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b45c597bd-7kwn5"] Jul 21 12:44:28.226214 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:28.226187 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b45c597bd-7kwn5"] Jul 21 12:44:29.928985 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:29.928949 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7ff3558-d86f-4e72-9e92-43e2c2354666" path="/var/lib/kubelet/pods/f7ff3558-d86f-4e72-9e92-43e2c2354666/volumes" Jul 21 12:44:37.791379 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:37.791353 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 12:44:37.796273 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:37.796248 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 12:44:37.797006 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:37.796981 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 12:44:37.801999 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:37.801962 2555 kubelet.go:1648] "Image garbage collection succeeded" Jul 21 12:44:37.802420 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:44:37.802404 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 12:45:15.509840 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.509803 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq"] Jul 21 12:45:15.511264 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.510124 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7ff3558-d86f-4e72-9e92-43e2c2354666" containerName="console" Jul 21 12:45:15.511264 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.510135 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7ff3558-d86f-4e72-9e92-43e2c2354666" containerName="console" Jul 21 12:45:15.511264 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.510207 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7ff3558-d86f-4e72-9e92-43e2c2354666" containerName="console" Jul 21 12:45:15.512170 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.512154 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq" Jul 21 12:45:15.514616 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.514574 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 21 12:45:15.514754 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.514574 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 21 12:45:15.515003 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.514989 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-j46d5\"" Jul 21 12:45:15.523739 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.523714 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq"] Jul 21 12:45:15.662859 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.662812 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3be8096-0b4b-41c2-b968-829667ae2300-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq\" (UID: \"c3be8096-0b4b-41c2-b968-829667ae2300\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq" Jul 21 12:45:15.662859 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.662873 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3be8096-0b4b-41c2-b968-829667ae2300-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq\" (UID: \"c3be8096-0b4b-41c2-b968-829667ae2300\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq" Jul 21 12:45:15.663086 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.662957 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jn52\" (UniqueName: \"kubernetes.io/projected/c3be8096-0b4b-41c2-b968-829667ae2300-kube-api-access-9jn52\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq\" (UID: \"c3be8096-0b4b-41c2-b968-829667ae2300\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq" Jul 21 12:45:15.763417 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.763316 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3be8096-0b4b-41c2-b968-829667ae2300-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq\" (UID: \"c3be8096-0b4b-41c2-b968-829667ae2300\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq" Jul 21 12:45:15.763417 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.763383 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jn52\" (UniqueName: \"kubernetes.io/projected/c3be8096-0b4b-41c2-b968-829667ae2300-kube-api-access-9jn52\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq\" (UID: \"c3be8096-0b4b-41c2-b968-829667ae2300\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq" Jul 21 12:45:15.763417 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.763423 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3be8096-0b4b-41c2-b968-829667ae2300-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq\" (UID: \"c3be8096-0b4b-41c2-b968-829667ae2300\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq" Jul 21 12:45:15.763784 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.763763 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3be8096-0b4b-41c2-b968-829667ae2300-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq\" (UID: \"c3be8096-0b4b-41c2-b968-829667ae2300\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq" Jul 21 12:45:15.763824 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.763792 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3be8096-0b4b-41c2-b968-829667ae2300-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq\" (UID: \"c3be8096-0b4b-41c2-b968-829667ae2300\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq" Jul 21 12:45:15.772982 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.772954 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jn52\" (UniqueName: \"kubernetes.io/projected/c3be8096-0b4b-41c2-b968-829667ae2300-kube-api-access-9jn52\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq\" (UID: \"c3be8096-0b4b-41c2-b968-829667ae2300\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq" Jul 21 12:45:15.821928 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.821891 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq" Jul 21 12:45:15.959472 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.959370 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq"] Jul 21 12:45:15.962323 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:45:15.962301 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3be8096_0b4b_41c2_b968_829667ae2300.slice/crio-982df41684afa46c0a1f14cdd00f051e13bc2fa3ae0c55eed9ce8fc7fcf889bf WatchSource:0}: Error finding container 982df41684afa46c0a1f14cdd00f051e13bc2fa3ae0c55eed9ce8fc7fcf889bf: Status 404 returned error can't find the container with id 982df41684afa46c0a1f14cdd00f051e13bc2fa3ae0c55eed9ce8fc7fcf889bf Jul 21 12:45:15.964707 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:15.964677 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 12:45:16.379682 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:16.379644 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq" event={"ID":"c3be8096-0b4b-41c2-b968-829667ae2300","Type":"ContainerStarted","Data":"982df41684afa46c0a1f14cdd00f051e13bc2fa3ae0c55eed9ce8fc7fcf889bf"} Jul 21 12:45:23.411180 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:23.411142 2555 generic.go:356] "Generic (PLEG): container finished" podID="c3be8096-0b4b-41c2-b968-829667ae2300" containerID="b89832d522e145ee6aaf6a915cf59a6604fc605dfd81beb81709bc8ff0e05e95" exitCode=0 Jul 21 12:45:23.411697 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:23.411230 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq" event={"ID":"c3be8096-0b4b-41c2-b968-829667ae2300","Type":"ContainerDied","Data":"b89832d522e145ee6aaf6a915cf59a6604fc605dfd81beb81709bc8ff0e05e95"} Jul 21 12:45:26.427260 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:26.427224 2555 generic.go:356] "Generic (PLEG): container finished" podID="c3be8096-0b4b-41c2-b968-829667ae2300" containerID="9b1b38e5c70a1259f4fdef0e1cda210fe160788b186e9939b6d32ed7a7b3f6b7" exitCode=0 Jul 21 12:45:26.427661 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:26.427289 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq" event={"ID":"c3be8096-0b4b-41c2-b968-829667ae2300","Type":"ContainerDied","Data":"9b1b38e5c70a1259f4fdef0e1cda210fe160788b186e9939b6d32ed7a7b3f6b7"} Jul 21 12:45:34.465847 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:34.465807 2555 generic.go:356] "Generic (PLEG): container finished" podID="c3be8096-0b4b-41c2-b968-829667ae2300" containerID="075414cc2880663d1f9ce74fcc863fa98f79f3cc5df1e6d1d15ac313b2c06b81" exitCode=0 Jul 21 12:45:34.466228 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:34.465846 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq" event={"ID":"c3be8096-0b4b-41c2-b968-829667ae2300","Type":"ContainerDied","Data":"075414cc2880663d1f9ce74fcc863fa98f79f3cc5df1e6d1d15ac313b2c06b81"} Jul 21 12:45:35.603285 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:35.603253 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq" Jul 21 12:45:35.618533 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:35.618502 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jn52\" (UniqueName: \"kubernetes.io/projected/c3be8096-0b4b-41c2-b968-829667ae2300-kube-api-access-9jn52\") pod \"c3be8096-0b4b-41c2-b968-829667ae2300\" (UID: \"c3be8096-0b4b-41c2-b968-829667ae2300\") " Jul 21 12:45:35.618703 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:35.618569 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3be8096-0b4b-41c2-b968-829667ae2300-util\") pod \"c3be8096-0b4b-41c2-b968-829667ae2300\" (UID: \"c3be8096-0b4b-41c2-b968-829667ae2300\") " Jul 21 12:45:35.618703 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:35.618654 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3be8096-0b4b-41c2-b968-829667ae2300-bundle\") pod \"c3be8096-0b4b-41c2-b968-829667ae2300\" (UID: \"c3be8096-0b4b-41c2-b968-829667ae2300\") " Jul 21 12:45:35.619329 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:35.619275 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3be8096-0b4b-41c2-b968-829667ae2300-bundle" (OuterVolumeSpecName: "bundle") pod "c3be8096-0b4b-41c2-b968-829667ae2300" (UID: "c3be8096-0b4b-41c2-b968-829667ae2300"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:45:35.620944 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:35.620911 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3be8096-0b4b-41c2-b968-829667ae2300-kube-api-access-9jn52" (OuterVolumeSpecName: "kube-api-access-9jn52") pod "c3be8096-0b4b-41c2-b968-829667ae2300" (UID: "c3be8096-0b4b-41c2-b968-829667ae2300"). InnerVolumeSpecName "kube-api-access-9jn52". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:45:35.623274 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:35.623249 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3be8096-0b4b-41c2-b968-829667ae2300-util" (OuterVolumeSpecName: "util") pod "c3be8096-0b4b-41c2-b968-829667ae2300" (UID: "c3be8096-0b4b-41c2-b968-829667ae2300"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:45:35.720060 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:35.720022 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jn52\" (UniqueName: \"kubernetes.io/projected/c3be8096-0b4b-41c2-b968-829667ae2300-kube-api-access-9jn52\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:45:35.720060 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:35.720049 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3be8096-0b4b-41c2-b968-829667ae2300-util\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:45:35.720060 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:35.720059 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3be8096-0b4b-41c2-b968-829667ae2300-bundle\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:45:36.475138 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:36.475107 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq" event={"ID":"c3be8096-0b4b-41c2-b968-829667ae2300","Type":"ContainerDied","Data":"982df41684afa46c0a1f14cdd00f051e13bc2fa3ae0c55eed9ce8fc7fcf889bf"} Jul 21 12:45:36.475138 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:36.475132 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540tjskq" Jul 21 12:45:36.475352 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:36.475153 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="982df41684afa46c0a1f14cdd00f051e13bc2fa3ae0c55eed9ce8fc7fcf889bf" Jul 21 12:45:43.673876 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.673837 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgs7b"] Jul 21 12:45:43.674324 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.674154 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3be8096-0b4b-41c2-b968-829667ae2300" containerName="util" Jul 21 12:45:43.674324 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.674166 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3be8096-0b4b-41c2-b968-829667ae2300" containerName="util" Jul 21 12:45:43.674324 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.674172 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3be8096-0b4b-41c2-b968-829667ae2300" containerName="pull" Jul 21 12:45:43.674324 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.674177 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3be8096-0b4b-41c2-b968-829667ae2300" containerName="pull" Jul 21 12:45:43.674324 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.674191 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3be8096-0b4b-41c2-b968-829667ae2300" containerName="extract" Jul 21 12:45:43.674324 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.674200 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3be8096-0b4b-41c2-b968-829667ae2300" containerName="extract" Jul 21 12:45:43.674324 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.674251 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3be8096-0b4b-41c2-b968-829667ae2300" containerName="extract" Jul 21 12:45:43.677684 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.677667 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgs7b" Jul 21 12:45:43.680412 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.680390 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-j5wcx\"" Jul 21 12:45:43.680640 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.680558 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 21 12:45:43.680702 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.680675 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 21 12:45:43.692066 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.692044 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgs7b"] Jul 21 12:45:43.779301 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.779260 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f44b989f-4337-4de9-a027-62259ba7cdea-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rgs7b\" (UID: \"f44b989f-4337-4de9-a027-62259ba7cdea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgs7b" Jul 21 12:45:43.779301 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.779309 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtq49\" (UniqueName: \"kubernetes.io/projected/f44b989f-4337-4de9-a027-62259ba7cdea-kube-api-access-gtq49\") pod \"cert-manager-operator-controller-manager-55c444bf86-rgs7b\" (UID: \"f44b989f-4337-4de9-a027-62259ba7cdea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgs7b" Jul 21 12:45:43.880353 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.880314 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f44b989f-4337-4de9-a027-62259ba7cdea-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rgs7b\" (UID: \"f44b989f-4337-4de9-a027-62259ba7cdea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgs7b" Jul 21 12:45:43.880520 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.880362 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtq49\" (UniqueName: \"kubernetes.io/projected/f44b989f-4337-4de9-a027-62259ba7cdea-kube-api-access-gtq49\") pod \"cert-manager-operator-controller-manager-55c444bf86-rgs7b\" (UID: \"f44b989f-4337-4de9-a027-62259ba7cdea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgs7b" Jul 21 12:45:43.880723 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.880703 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f44b989f-4337-4de9-a027-62259ba7cdea-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rgs7b\" (UID: \"f44b989f-4337-4de9-a027-62259ba7cdea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgs7b" Jul 21 12:45:43.890187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.890157 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtq49\" (UniqueName: \"kubernetes.io/projected/f44b989f-4337-4de9-a027-62259ba7cdea-kube-api-access-gtq49\") pod \"cert-manager-operator-controller-manager-55c444bf86-rgs7b\" (UID: \"f44b989f-4337-4de9-a027-62259ba7cdea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgs7b" Jul 21 12:45:43.986978 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:43.986872 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgs7b" Jul 21 12:45:44.133532 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:44.133493 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgs7b"] Jul 21 12:45:44.137055 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:45:44.137025 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf44b989f_4337_4de9_a027_62259ba7cdea.slice/crio-2832b007bff784756929061f86d5e8543e3d4028ca2cfdca3fbc0d7b7f1399b7 WatchSource:0}: Error finding container 2832b007bff784756929061f86d5e8543e3d4028ca2cfdca3fbc0d7b7f1399b7: Status 404 returned error can't find the container with id 2832b007bff784756929061f86d5e8543e3d4028ca2cfdca3fbc0d7b7f1399b7 Jul 21 12:45:44.506266 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:44.506226 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgs7b" event={"ID":"f44b989f-4337-4de9-a027-62259ba7cdea","Type":"ContainerStarted","Data":"2832b007bff784756929061f86d5e8543e3d4028ca2cfdca3fbc0d7b7f1399b7"} Jul 21 12:45:46.516101 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:46.516013 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgs7b" event={"ID":"f44b989f-4337-4de9-a027-62259ba7cdea","Type":"ContainerStarted","Data":"591d2d26aa9b1357f4c9846308bb19e2477750021b09a597bf44089b1ef98e21"} Jul 21 12:45:46.542843 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:46.542781 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rgs7b" podStartSLOduration=2.046386776 podStartE2EDuration="3.542760482s" totalImagesPullingTime="1.496373706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:45:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:45:46.541343426 +0000 UTC m=+369.159636373" watchObservedRunningTime="2026-07-21 12:45:46.542760482 +0000 UTC m=+369.161053430" Jul 21 12:45:46.570685 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:46.570646 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-brk96"] Jul 21 12:45:46.572968 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:46.572946 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-brk96" Jul 21 12:45:46.575235 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:46.575215 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-hlw85\"" Jul 21 12:45:46.575425 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:46.575405 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 21 12:45:46.575544 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:46.575436 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 21 12:45:46.588656 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:46.588629 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-brk96"] Jul 21 12:45:46.604100 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:46.604070 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/469a7aa7-e762-49dc-b482-4b7d9ac5f168-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-brk96\" (UID: \"469a7aa7-e762-49dc-b482-4b7d9ac5f168\") " pod="cert-manager/cert-manager-webhook-6486ffd997-brk96" Jul 21 12:45:46.604264 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:46.604123 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trrrr\" (UniqueName: \"kubernetes.io/projected/469a7aa7-e762-49dc-b482-4b7d9ac5f168-kube-api-access-trrrr\") pod \"cert-manager-webhook-6486ffd997-brk96\" (UID: \"469a7aa7-e762-49dc-b482-4b7d9ac5f168\") " pod="cert-manager/cert-manager-webhook-6486ffd997-brk96" Jul 21 12:45:46.705395 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:46.705360 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/469a7aa7-e762-49dc-b482-4b7d9ac5f168-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-brk96\" (UID: \"469a7aa7-e762-49dc-b482-4b7d9ac5f168\") " pod="cert-manager/cert-manager-webhook-6486ffd997-brk96" Jul 21 12:45:46.705587 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:46.705412 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trrrr\" (UniqueName: \"kubernetes.io/projected/469a7aa7-e762-49dc-b482-4b7d9ac5f168-kube-api-access-trrrr\") pod \"cert-manager-webhook-6486ffd997-brk96\" (UID: \"469a7aa7-e762-49dc-b482-4b7d9ac5f168\") " pod="cert-manager/cert-manager-webhook-6486ffd997-brk96" Jul 21 12:45:46.716968 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:46.716932 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/469a7aa7-e762-49dc-b482-4b7d9ac5f168-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-brk96\" (UID: \"469a7aa7-e762-49dc-b482-4b7d9ac5f168\") " pod="cert-manager/cert-manager-webhook-6486ffd997-brk96" Jul 21 12:45:46.717111 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:46.717009 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trrrr\" (UniqueName: \"kubernetes.io/projected/469a7aa7-e762-49dc-b482-4b7d9ac5f168-kube-api-access-trrrr\") pod \"cert-manager-webhook-6486ffd997-brk96\" (UID: \"469a7aa7-e762-49dc-b482-4b7d9ac5f168\") " pod="cert-manager/cert-manager-webhook-6486ffd997-brk96" Jul 21 12:45:46.892763 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:46.892725 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-brk96" Jul 21 12:45:47.052268 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:47.052229 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-brk96"] Jul 21 12:45:47.056103 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:45:47.056070 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod469a7aa7_e762_49dc_b482_4b7d9ac5f168.slice/crio-13f0a4463b6a123dd95100fb0b5b23fc7408503f74be11eeceb499998d9617e8 WatchSource:0}: Error finding container 13f0a4463b6a123dd95100fb0b5b23fc7408503f74be11eeceb499998d9617e8: Status 404 returned error can't find the container with id 13f0a4463b6a123dd95100fb0b5b23fc7408503f74be11eeceb499998d9617e8 Jul 21 12:45:47.521244 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:47.521210 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-brk96" event={"ID":"469a7aa7-e762-49dc-b482-4b7d9ac5f168","Type":"ContainerStarted","Data":"13f0a4463b6a123dd95100fb0b5b23fc7408503f74be11eeceb499998d9617e8"} Jul 21 12:45:48.072381 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:48.072339 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp"] Jul 21 12:45:48.076246 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:48.076226 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp" Jul 21 12:45:48.078693 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:48.078667 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 21 12:45:48.078830 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:48.078703 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 21 12:45:48.079134 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:48.079120 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-j46d5\"" Jul 21 12:45:48.089651 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:48.089620 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp"] Jul 21 12:45:48.116662 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:48.116622 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d18e9c8-ebd2-4630-804b-745f722140a4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp\" (UID: \"3d18e9c8-ebd2-4630-804b-745f722140a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp" Jul 21 12:45:48.116837 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:48.116673 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg4p6\" (UniqueName: \"kubernetes.io/projected/3d18e9c8-ebd2-4630-804b-745f722140a4-kube-api-access-sg4p6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp\" (UID: \"3d18e9c8-ebd2-4630-804b-745f722140a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp" Jul 21 12:45:48.116837 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:48.116701 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d18e9c8-ebd2-4630-804b-745f722140a4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp\" (UID: \"3d18e9c8-ebd2-4630-804b-745f722140a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp" Jul 21 12:45:48.217242 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:48.217204 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg4p6\" (UniqueName: \"kubernetes.io/projected/3d18e9c8-ebd2-4630-804b-745f722140a4-kube-api-access-sg4p6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp\" (UID: \"3d18e9c8-ebd2-4630-804b-745f722140a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp" Jul 21 12:45:48.217406 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:48.217258 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d18e9c8-ebd2-4630-804b-745f722140a4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp\" (UID: \"3d18e9c8-ebd2-4630-804b-745f722140a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp" Jul 21 12:45:48.217406 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:48.217336 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d18e9c8-ebd2-4630-804b-745f722140a4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp\" (UID: \"3d18e9c8-ebd2-4630-804b-745f722140a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp" Jul 21 12:45:48.217729 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:48.217712 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d18e9c8-ebd2-4630-804b-745f722140a4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp\" (UID: \"3d18e9c8-ebd2-4630-804b-745f722140a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp" Jul 21 12:45:48.217771 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:48.217723 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d18e9c8-ebd2-4630-804b-745f722140a4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp\" (UID: \"3d18e9c8-ebd2-4630-804b-745f722140a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp" Jul 21 12:45:48.227666 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:48.227632 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg4p6\" (UniqueName: \"kubernetes.io/projected/3d18e9c8-ebd2-4630-804b-745f722140a4-kube-api-access-sg4p6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp\" (UID: \"3d18e9c8-ebd2-4630-804b-745f722140a4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp" Jul 21 12:45:48.386370 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:48.386338 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp" Jul 21 12:45:48.524702 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:48.524668 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp"] Jul 21 12:45:48.528663 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:45:48.528628 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d18e9c8_ebd2_4630_804b_745f722140a4.slice/crio-8189dab43a935726fea1c2ae23d377cf0535742e884930fa919fd44a50e417d0 WatchSource:0}: Error finding container 8189dab43a935726fea1c2ae23d377cf0535742e884930fa919fd44a50e417d0: Status 404 returned error can't find the container with id 8189dab43a935726fea1c2ae23d377cf0535742e884930fa919fd44a50e417d0 Jul 21 12:45:49.535608 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:49.535500 2555 generic.go:356] "Generic (PLEG): container finished" podID="3d18e9c8-ebd2-4630-804b-745f722140a4" containerID="d5f8546dc8ee82fbe3ceb14170431609abed6a4a6620ccfb508f9fa8c8620b40" exitCode=0 Jul 21 12:45:49.535988 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:49.535584 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp" event={"ID":"3d18e9c8-ebd2-4630-804b-745f722140a4","Type":"ContainerDied","Data":"d5f8546dc8ee82fbe3ceb14170431609abed6a4a6620ccfb508f9fa8c8620b40"} Jul 21 12:45:49.535988 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:49.535687 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp" event={"ID":"3d18e9c8-ebd2-4630-804b-745f722140a4","Type":"ContainerStarted","Data":"8189dab43a935726fea1c2ae23d377cf0535742e884930fa919fd44a50e417d0"} Jul 21 12:45:57.144369 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:57.144335 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-88lfg"] Jul 21 12:45:57.148892 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:57.148871 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-88lfg" Jul 21 12:45:57.151123 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:57.151098 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-sdwvt\"" Jul 21 12:45:57.158725 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:57.158701 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-88lfg"] Jul 21 12:45:57.293173 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:57.293131 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/26480dcd-151a-4a24-b092-50bcddd95737-bound-sa-token\") pod \"cert-manager-55b4d954d5-88lfg\" (UID: \"26480dcd-151a-4a24-b092-50bcddd95737\") " pod="cert-manager/cert-manager-55b4d954d5-88lfg" Jul 21 12:45:57.293371 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:57.293189 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mtlg\" (UniqueName: \"kubernetes.io/projected/26480dcd-151a-4a24-b092-50bcddd95737-kube-api-access-6mtlg\") pod \"cert-manager-55b4d954d5-88lfg\" (UID: \"26480dcd-151a-4a24-b092-50bcddd95737\") " pod="cert-manager/cert-manager-55b4d954d5-88lfg" Jul 21 12:45:57.394328 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:57.394286 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mtlg\" (UniqueName: \"kubernetes.io/projected/26480dcd-151a-4a24-b092-50bcddd95737-kube-api-access-6mtlg\") pod \"cert-manager-55b4d954d5-88lfg\" (UID: \"26480dcd-151a-4a24-b092-50bcddd95737\") " pod="cert-manager/cert-manager-55b4d954d5-88lfg" Jul 21 12:45:57.394568 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:57.394393 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/26480dcd-151a-4a24-b092-50bcddd95737-bound-sa-token\") pod \"cert-manager-55b4d954d5-88lfg\" (UID: \"26480dcd-151a-4a24-b092-50bcddd95737\") " pod="cert-manager/cert-manager-55b4d954d5-88lfg" Jul 21 12:45:57.406799 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:57.406772 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/26480dcd-151a-4a24-b092-50bcddd95737-bound-sa-token\") pod \"cert-manager-55b4d954d5-88lfg\" (UID: \"26480dcd-151a-4a24-b092-50bcddd95737\") " pod="cert-manager/cert-manager-55b4d954d5-88lfg" Jul 21 12:45:57.407020 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:57.407000 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mtlg\" (UniqueName: \"kubernetes.io/projected/26480dcd-151a-4a24-b092-50bcddd95737-kube-api-access-6mtlg\") pod \"cert-manager-55b4d954d5-88lfg\" (UID: \"26480dcd-151a-4a24-b092-50bcddd95737\") " pod="cert-manager/cert-manager-55b4d954d5-88lfg" Jul 21 12:45:57.458159 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:57.458119 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-88lfg" Jul 21 12:45:57.598586 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:57.598554 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-88lfg"] Jul 21 12:45:57.601424 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:45:57.601389 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26480dcd_151a_4a24_b092_50bcddd95737.slice/crio-870cdea6161888e005adfc8b20973b9bc793536be126dc00134f4b3e4ba7b2b1 WatchSource:0}: Error finding container 870cdea6161888e005adfc8b20973b9bc793536be126dc00134f4b3e4ba7b2b1: Status 404 returned error can't find the container with id 870cdea6161888e005adfc8b20973b9bc793536be126dc00134f4b3e4ba7b2b1 Jul 21 12:45:58.579773 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:45:58.579735 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-88lfg" event={"ID":"26480dcd-151a-4a24-b092-50bcddd95737","Type":"ContainerStarted","Data":"870cdea6161888e005adfc8b20973b9bc793536be126dc00134f4b3e4ba7b2b1"} Jul 21 12:46:02.599666 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:02.599630 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-brk96" event={"ID":"469a7aa7-e762-49dc-b482-4b7d9ac5f168","Type":"ContainerStarted","Data":"c158b146cbeabc8a9bced937dd9cb29d95da60d927f2847d7f8e55b182dc117f"} Jul 21 12:46:02.600116 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:02.599752 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-brk96" Jul 21 12:46:02.601139 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:02.601117 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-88lfg" event={"ID":"26480dcd-151a-4a24-b092-50bcddd95737","Type":"ContainerStarted","Data":"fcffef1aeb5fee2873a33fe0a993d3f800a2be575fecc774f8991e3e82b33d31"} Jul 21 12:46:02.602819 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:02.602798 2555 generic.go:356] "Generic (PLEG): container finished" podID="3d18e9c8-ebd2-4630-804b-745f722140a4" containerID="501d9ce0bbbfa33b15740d5a9646156629f0cd93e663b3791187193c10215a50" exitCode=0 Jul 21 12:46:02.602911 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:02.602860 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp" event={"ID":"3d18e9c8-ebd2-4630-804b-745f722140a4","Type":"ContainerDied","Data":"501d9ce0bbbfa33b15740d5a9646156629f0cd93e663b3791187193c10215a50"} Jul 21 12:46:02.616399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:02.616347 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-brk96" podStartSLOduration=2.173105659 podStartE2EDuration="16.616332019s" totalImagesPullingTime="14.44322636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:45:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:46:02.615409895 +0000 UTC m=+385.233702845" watchObservedRunningTime="2026-07-21 12:46:02.616332019 +0000 UTC m=+385.234624968" Jul 21 12:46:02.630704 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:02.630654 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-88lfg" podStartSLOduration=1.726565304 podStartE2EDuration="5.630635921s" totalImagesPullingTime="3.904070617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:45:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:46:02.630302981 +0000 UTC m=+385.248595931" watchObservedRunningTime="2026-07-21 12:46:02.630635921 +0000 UTC m=+385.248928869" Jul 21 12:46:03.609564 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:03.609531 2555 generic.go:356] "Generic (PLEG): container finished" podID="3d18e9c8-ebd2-4630-804b-745f722140a4" containerID="5ca7741f459c7ba99cface1c132b1ffb7409ad875ed14bf0f154981b4da9d4a8" exitCode=0 Jul 21 12:46:03.610009 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:03.609622 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp" event={"ID":"3d18e9c8-ebd2-4630-804b-745f722140a4","Type":"ContainerDied","Data":"5ca7741f459c7ba99cface1c132b1ffb7409ad875ed14bf0f154981b4da9d4a8"} Jul 21 12:46:04.747450 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:04.747426 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp" Jul 21 12:46:04.860115 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:04.860078 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d18e9c8-ebd2-4630-804b-745f722140a4-util\") pod \"3d18e9c8-ebd2-4630-804b-745f722140a4\" (UID: \"3d18e9c8-ebd2-4630-804b-745f722140a4\") " Jul 21 12:46:04.860314 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:04.860170 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sg4p6\" (UniqueName: \"kubernetes.io/projected/3d18e9c8-ebd2-4630-804b-745f722140a4-kube-api-access-sg4p6\") pod \"3d18e9c8-ebd2-4630-804b-745f722140a4\" (UID: \"3d18e9c8-ebd2-4630-804b-745f722140a4\") " Jul 21 12:46:04.860314 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:04.860223 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d18e9c8-ebd2-4630-804b-745f722140a4-bundle\") pod \"3d18e9c8-ebd2-4630-804b-745f722140a4\" (UID: \"3d18e9c8-ebd2-4630-804b-745f722140a4\") " Jul 21 12:46:04.860692 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:04.860665 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d18e9c8-ebd2-4630-804b-745f722140a4-bundle" (OuterVolumeSpecName: "bundle") pod "3d18e9c8-ebd2-4630-804b-745f722140a4" (UID: "3d18e9c8-ebd2-4630-804b-745f722140a4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:46:04.862420 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:04.862384 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d18e9c8-ebd2-4630-804b-745f722140a4-kube-api-access-sg4p6" (OuterVolumeSpecName: "kube-api-access-sg4p6") pod "3d18e9c8-ebd2-4630-804b-745f722140a4" (UID: "3d18e9c8-ebd2-4630-804b-745f722140a4"). InnerVolumeSpecName "kube-api-access-sg4p6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:46:04.864566 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:04.864519 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d18e9c8-ebd2-4630-804b-745f722140a4-util" (OuterVolumeSpecName: "util") pod "3d18e9c8-ebd2-4630-804b-745f722140a4" (UID: "3d18e9c8-ebd2-4630-804b-745f722140a4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:46:04.961521 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:04.961474 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d18e9c8-ebd2-4630-804b-745f722140a4-util\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:46:04.961521 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:04.961510 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sg4p6\" (UniqueName: \"kubernetes.io/projected/3d18e9c8-ebd2-4630-804b-745f722140a4-kube-api-access-sg4p6\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:46:04.961521 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:04.961519 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d18e9c8-ebd2-4630-804b-745f722140a4-bundle\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:46:05.619796 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:05.619764 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp" event={"ID":"3d18e9c8-ebd2-4630-804b-745f722140a4","Type":"ContainerDied","Data":"8189dab43a935726fea1c2ae23d377cf0535742e884930fa919fd44a50e417d0"} Jul 21 12:46:05.619796 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:05.619785 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f98xmp" Jul 21 12:46:05.620010 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:05.619809 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8189dab43a935726fea1c2ae23d377cf0535742e884930fa919fd44a50e417d0" Jul 21 12:46:08.612474 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:08.612442 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-brk96" Jul 21 12:46:16.249871 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.249835 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln"] Jul 21 12:46:16.250279 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.250131 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d18e9c8-ebd2-4630-804b-745f722140a4" containerName="extract" Jul 21 12:46:16.250279 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.250142 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d18e9c8-ebd2-4630-804b-745f722140a4" containerName="extract" Jul 21 12:46:16.250279 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.250158 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d18e9c8-ebd2-4630-804b-745f722140a4" containerName="pull" Jul 21 12:46:16.250279 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.250163 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d18e9c8-ebd2-4630-804b-745f722140a4" containerName="pull" Jul 21 12:46:16.250279 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.250172 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d18e9c8-ebd2-4630-804b-745f722140a4" containerName="util" Jul 21 12:46:16.250279 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.250177 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d18e9c8-ebd2-4630-804b-745f722140a4" containerName="util" Jul 21 12:46:16.250279 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.250239 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d18e9c8-ebd2-4630-804b-745f722140a4" containerName="extract" Jul 21 12:46:16.252605 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.252578 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln" Jul 21 12:46:16.254780 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.254755 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 21 12:46:16.254780 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.254766 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-j46d5\"" Jul 21 12:46:16.255325 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.255311 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 21 12:46:16.264619 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.264578 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln"] Jul 21 12:46:16.351910 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.351875 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab117-2558-4199-a169-cb9cce900a23-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln\" (UID: \"fa6ab117-2558-4199-a169-cb9cce900a23\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln" Jul 21 12:46:16.351910 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.351913 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz2nc\" (UniqueName: \"kubernetes.io/projected/fa6ab117-2558-4199-a169-cb9cce900a23-kube-api-access-tz2nc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln\" (UID: \"fa6ab117-2558-4199-a169-cb9cce900a23\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln" Jul 21 12:46:16.352137 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.351939 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab117-2558-4199-a169-cb9cce900a23-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln\" (UID: \"fa6ab117-2558-4199-a169-cb9cce900a23\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln" Jul 21 12:46:16.453369 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.453334 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab117-2558-4199-a169-cb9cce900a23-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln\" (UID: \"fa6ab117-2558-4199-a169-cb9cce900a23\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln" Jul 21 12:46:16.453369 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.453372 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz2nc\" (UniqueName: \"kubernetes.io/projected/fa6ab117-2558-4199-a169-cb9cce900a23-kube-api-access-tz2nc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln\" (UID: \"fa6ab117-2558-4199-a169-cb9cce900a23\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln" Jul 21 12:46:16.453683 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.453395 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab117-2558-4199-a169-cb9cce900a23-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln\" (UID: \"fa6ab117-2558-4199-a169-cb9cce900a23\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln" Jul 21 12:46:16.453875 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.453852 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab117-2558-4199-a169-cb9cce900a23-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln\" (UID: \"fa6ab117-2558-4199-a169-cb9cce900a23\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln" Jul 21 12:46:16.453944 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.453882 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab117-2558-4199-a169-cb9cce900a23-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln\" (UID: \"fa6ab117-2558-4199-a169-cb9cce900a23\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln" Jul 21 12:46:16.463792 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.463766 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz2nc\" (UniqueName: \"kubernetes.io/projected/fa6ab117-2558-4199-a169-cb9cce900a23-kube-api-access-tz2nc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln\" (UID: \"fa6ab117-2558-4199-a169-cb9cce900a23\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln" Jul 21 12:46:16.562091 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.562002 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln" Jul 21 12:46:16.700170 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:16.700134 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln"] Jul 21 12:46:16.703561 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:46:16.703531 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa6ab117_2558_4199_a169_cb9cce900a23.slice/crio-371711ca6e0d943906320b062abb3d52d57aea37446d8fd8b68796db7a2ad32a WatchSource:0}: Error finding container 371711ca6e0d943906320b062abb3d52d57aea37446d8fd8b68796db7a2ad32a: Status 404 returned error can't find the container with id 371711ca6e0d943906320b062abb3d52d57aea37446d8fd8b68796db7a2ad32a Jul 21 12:46:17.676086 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:17.676050 2555 generic.go:356] "Generic (PLEG): container finished" podID="fa6ab117-2558-4199-a169-cb9cce900a23" containerID="5c804d96b0552ca4ca0ec5a77bf295b87e0e3aa3749db5dfdf66b8a0dcf1f1c0" exitCode=0 Jul 21 12:46:17.676468 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:17.676142 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln" event={"ID":"fa6ab117-2558-4199-a169-cb9cce900a23","Type":"ContainerDied","Data":"5c804d96b0552ca4ca0ec5a77bf295b87e0e3aa3749db5dfdf66b8a0dcf1f1c0"} Jul 21 12:46:17.676468 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:17.676198 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln" event={"ID":"fa6ab117-2558-4199-a169-cb9cce900a23","Type":"ContainerStarted","Data":"371711ca6e0d943906320b062abb3d52d57aea37446d8fd8b68796db7a2ad32a"} Jul 21 12:46:18.686979 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:18.686945 2555 generic.go:356] "Generic (PLEG): container finished" podID="fa6ab117-2558-4199-a169-cb9cce900a23" containerID="fc5c895140bbee8e76e2ef0e7d1b475033e3ab88e279ed10776d662b6351404b" exitCode=0 Jul 21 12:46:18.687433 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:18.687009 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln" event={"ID":"fa6ab117-2558-4199-a169-cb9cce900a23","Type":"ContainerDied","Data":"fc5c895140bbee8e76e2ef0e7d1b475033e3ab88e279ed10776d662b6351404b"} Jul 21 12:46:19.694675 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:19.694104 2555 generic.go:356] "Generic (PLEG): container finished" podID="fa6ab117-2558-4199-a169-cb9cce900a23" containerID="880cc75681327982d4ae44642c18dbbd63cb42fbd5da40ef7b74492854de6521" exitCode=0 Jul 21 12:46:19.694675 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:19.694252 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln" event={"ID":"fa6ab117-2558-4199-a169-cb9cce900a23","Type":"ContainerDied","Data":"880cc75681327982d4ae44642c18dbbd63cb42fbd5da40ef7b74492854de6521"} Jul 21 12:46:20.826277 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:20.826249 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln" Jul 21 12:46:20.989696 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:20.989583 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab117-2558-4199-a169-cb9cce900a23-util\") pod \"fa6ab117-2558-4199-a169-cb9cce900a23\" (UID: \"fa6ab117-2558-4199-a169-cb9cce900a23\") " Jul 21 12:46:20.989696 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:20.989647 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tz2nc\" (UniqueName: \"kubernetes.io/projected/fa6ab117-2558-4199-a169-cb9cce900a23-kube-api-access-tz2nc\") pod \"fa6ab117-2558-4199-a169-cb9cce900a23\" (UID: \"fa6ab117-2558-4199-a169-cb9cce900a23\") " Jul 21 12:46:20.989918 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:20.989714 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab117-2558-4199-a169-cb9cce900a23-bundle\") pod \"fa6ab117-2558-4199-a169-cb9cce900a23\" (UID: \"fa6ab117-2558-4199-a169-cb9cce900a23\") " Jul 21 12:46:20.990462 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:20.990437 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa6ab117-2558-4199-a169-cb9cce900a23-bundle" (OuterVolumeSpecName: "bundle") pod "fa6ab117-2558-4199-a169-cb9cce900a23" (UID: "fa6ab117-2558-4199-a169-cb9cce900a23"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:46:20.991934 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:20.991902 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa6ab117-2558-4199-a169-cb9cce900a23-kube-api-access-tz2nc" (OuterVolumeSpecName: "kube-api-access-tz2nc") pod "fa6ab117-2558-4199-a169-cb9cce900a23" (UID: "fa6ab117-2558-4199-a169-cb9cce900a23"). InnerVolumeSpecName "kube-api-access-tz2nc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:46:20.995766 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:20.995718 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa6ab117-2558-4199-a169-cb9cce900a23-util" (OuterVolumeSpecName: "util") pod "fa6ab117-2558-4199-a169-cb9cce900a23" (UID: "fa6ab117-2558-4199-a169-cb9cce900a23"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:46:21.090744 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:21.090709 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab117-2558-4199-a169-cb9cce900a23-bundle\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:46:21.090744 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:21.090737 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab117-2558-4199-a169-cb9cce900a23-util\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:46:21.090744 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:21.090745 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tz2nc\" (UniqueName: \"kubernetes.io/projected/fa6ab117-2558-4199-a169-cb9cce900a23-kube-api-access-tz2nc\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:46:21.705986 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:21.705954 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln" event={"ID":"fa6ab117-2558-4199-a169-cb9cce900a23","Type":"ContainerDied","Data":"371711ca6e0d943906320b062abb3d52d57aea37446d8fd8b68796db7a2ad32a"} Jul 21 12:46:21.706161 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:21.705998 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afn7rln" Jul 21 12:46:21.706161 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:21.706002 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="371711ca6e0d943906320b062abb3d52d57aea37446d8fd8b68796db7a2ad32a" Jul 21 12:46:23.356688 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.356636 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t"] Jul 21 12:46:23.357131 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.356949 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa6ab117-2558-4199-a169-cb9cce900a23" containerName="extract" Jul 21 12:46:23.357131 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.356960 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa6ab117-2558-4199-a169-cb9cce900a23" containerName="extract" Jul 21 12:46:23.357131 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.356974 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa6ab117-2558-4199-a169-cb9cce900a23" containerName="util" Jul 21 12:46:23.357131 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.356980 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa6ab117-2558-4199-a169-cb9cce900a23" containerName="util" Jul 21 12:46:23.357131 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.356988 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa6ab117-2558-4199-a169-cb9cce900a23" containerName="pull" Jul 21 12:46:23.357131 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.356995 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa6ab117-2558-4199-a169-cb9cce900a23" containerName="pull" Jul 21 12:46:23.357131 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.357042 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa6ab117-2558-4199-a169-cb9cce900a23" containerName="extract" Jul 21 12:46:23.359018 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.359003 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t" Jul 21 12:46:23.361798 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.361771 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 21 12:46:23.361954 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.361823 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 21 12:46:23.361954 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.361833 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 21 12:46:23.361954 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.361889 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vjqqc\"" Jul 21 12:46:23.361954 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.361906 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 21 12:46:23.361954 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.361893 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 21 12:46:23.371737 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.371711 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t"] Jul 21 12:46:23.510477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.510440 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/590aa5c3-3489-4866-ace2-1cb44b5d3851-cert\") pod \"lws-controller-manager-84bfb648b5-sfv9t\" (UID: \"590aa5c3-3489-4866-ace2-1cb44b5d3851\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t" Jul 21 12:46:23.510687 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.510494 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/590aa5c3-3489-4866-ace2-1cb44b5d3851-metrics-cert\") pod \"lws-controller-manager-84bfb648b5-sfv9t\" (UID: \"590aa5c3-3489-4866-ace2-1cb44b5d3851\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t" Jul 21 12:46:23.510687 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.510521 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pjbc\" (UniqueName: \"kubernetes.io/projected/590aa5c3-3489-4866-ace2-1cb44b5d3851-kube-api-access-9pjbc\") pod \"lws-controller-manager-84bfb648b5-sfv9t\" (UID: \"590aa5c3-3489-4866-ace2-1cb44b5d3851\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t" Jul 21 12:46:23.510687 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.510543 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/590aa5c3-3489-4866-ace2-1cb44b5d3851-manager-config\") pod \"lws-controller-manager-84bfb648b5-sfv9t\" (UID: \"590aa5c3-3489-4866-ace2-1cb44b5d3851\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t" Jul 21 12:46:23.612011 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.611926 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/590aa5c3-3489-4866-ace2-1cb44b5d3851-cert\") pod \"lws-controller-manager-84bfb648b5-sfv9t\" (UID: \"590aa5c3-3489-4866-ace2-1cb44b5d3851\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t" Jul 21 12:46:23.612011 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.611981 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/590aa5c3-3489-4866-ace2-1cb44b5d3851-metrics-cert\") pod \"lws-controller-manager-84bfb648b5-sfv9t\" (UID: \"590aa5c3-3489-4866-ace2-1cb44b5d3851\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t" Jul 21 12:46:23.612241 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.612023 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pjbc\" (UniqueName: \"kubernetes.io/projected/590aa5c3-3489-4866-ace2-1cb44b5d3851-kube-api-access-9pjbc\") pod \"lws-controller-manager-84bfb648b5-sfv9t\" (UID: \"590aa5c3-3489-4866-ace2-1cb44b5d3851\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t" Jul 21 12:46:23.612241 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.612051 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/590aa5c3-3489-4866-ace2-1cb44b5d3851-manager-config\") pod \"lws-controller-manager-84bfb648b5-sfv9t\" (UID: \"590aa5c3-3489-4866-ace2-1cb44b5d3851\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t" Jul 21 12:46:23.612719 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.612696 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/590aa5c3-3489-4866-ace2-1cb44b5d3851-manager-config\") pod \"lws-controller-manager-84bfb648b5-sfv9t\" (UID: \"590aa5c3-3489-4866-ace2-1cb44b5d3851\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t" Jul 21 12:46:23.614557 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.614535 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/590aa5c3-3489-4866-ace2-1cb44b5d3851-metrics-cert\") pod \"lws-controller-manager-84bfb648b5-sfv9t\" (UID: \"590aa5c3-3489-4866-ace2-1cb44b5d3851\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t" Jul 21 12:46:23.614674 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.614565 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/590aa5c3-3489-4866-ace2-1cb44b5d3851-cert\") pod \"lws-controller-manager-84bfb648b5-sfv9t\" (UID: \"590aa5c3-3489-4866-ace2-1cb44b5d3851\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t" Jul 21 12:46:23.621843 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.621820 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pjbc\" (UniqueName: \"kubernetes.io/projected/590aa5c3-3489-4866-ace2-1cb44b5d3851-kube-api-access-9pjbc\") pod \"lws-controller-manager-84bfb648b5-sfv9t\" (UID: \"590aa5c3-3489-4866-ace2-1cb44b5d3851\") " pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t" Jul 21 12:46:23.668817 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.668780 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t" Jul 21 12:46:23.817423 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:23.817387 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t"] Jul 21 12:46:23.822088 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:46:23.822061 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod590aa5c3_3489_4866_ace2_1cb44b5d3851.slice/crio-dc0a204436a178bd1ef1c7d427992c4aeeeb0a5f6ce2516aad4eb2fdf8373986 WatchSource:0}: Error finding container dc0a204436a178bd1ef1c7d427992c4aeeeb0a5f6ce2516aad4eb2fdf8373986: Status 404 returned error can't find the container with id dc0a204436a178bd1ef1c7d427992c4aeeeb0a5f6ce2516aad4eb2fdf8373986 Jul 21 12:46:24.722348 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:24.722309 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t" event={"ID":"590aa5c3-3489-4866-ace2-1cb44b5d3851","Type":"ContainerStarted","Data":"dc0a204436a178bd1ef1c7d427992c4aeeeb0a5f6ce2516aad4eb2fdf8373986"} Jul 21 12:46:25.728398 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:25.728360 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t" event={"ID":"590aa5c3-3489-4866-ace2-1cb44b5d3851","Type":"ContainerStarted","Data":"e499f54a38574125fb51c317c59662012a7026e08d349132ee93a96c53f27a99"} Jul 21 12:46:25.728849 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:25.728464 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t" Jul 21 12:46:25.746011 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:25.745953 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t" podStartSLOduration=1.30498925 podStartE2EDuration="2.745938028s" totalImagesPullingTime="1.440948778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:46:25.7447827 +0000 UTC m=+408.363075650" watchObservedRunningTime="2026-07-21 12:46:25.745938028 +0000 UTC m=+408.364230975" Jul 21 12:46:32.744075 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.744033 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b"] Jul 21 12:46:32.748937 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.748905 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" Jul 21 12:46:32.751105 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.751080 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 21 12:46:32.751281 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.751079 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 21 12:46:32.751281 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.751233 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bh5f5\"" Jul 21 12:46:32.751445 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.751399 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 21 12:46:32.751509 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.751451 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 21 12:46:32.765780 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.765737 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b"] Jul 21 12:46:32.789281 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.789242 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/82740182-3d7b-4d3c-8d49-8ed65c1ab143-manifests\") pod \"opendatahub-operator-controller-manager-579d89bddd-pq59b\" (UID: \"82740182-3d7b-4d3c-8d49-8ed65c1ab143\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" Jul 21 12:46:32.789281 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.789282 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82740182-3d7b-4d3c-8d49-8ed65c1ab143-tmp\") pod \"opendatahub-operator-controller-manager-579d89bddd-pq59b\" (UID: \"82740182-3d7b-4d3c-8d49-8ed65c1ab143\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" Jul 21 12:46:32.789526 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.789314 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/82740182-3d7b-4d3c-8d49-8ed65c1ab143-webhook-cert\") pod \"opendatahub-operator-controller-manager-579d89bddd-pq59b\" (UID: \"82740182-3d7b-4d3c-8d49-8ed65c1ab143\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" Jul 21 12:46:32.789526 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.789339 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/82740182-3d7b-4d3c-8d49-8ed65c1ab143-apiservice-cert\") pod \"opendatahub-operator-controller-manager-579d89bddd-pq59b\" (UID: \"82740182-3d7b-4d3c-8d49-8ed65c1ab143\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" Jul 21 12:46:32.789526 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.789355 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69pxf\" (UniqueName: \"kubernetes.io/projected/82740182-3d7b-4d3c-8d49-8ed65c1ab143-kube-api-access-69pxf\") pod \"opendatahub-operator-controller-manager-579d89bddd-pq59b\" (UID: \"82740182-3d7b-4d3c-8d49-8ed65c1ab143\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" Jul 21 12:46:32.889857 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.889811 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/82740182-3d7b-4d3c-8d49-8ed65c1ab143-manifests\") pod \"opendatahub-operator-controller-manager-579d89bddd-pq59b\" (UID: \"82740182-3d7b-4d3c-8d49-8ed65c1ab143\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" Jul 21 12:46:32.889857 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.889852 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82740182-3d7b-4d3c-8d49-8ed65c1ab143-tmp\") pod \"opendatahub-operator-controller-manager-579d89bddd-pq59b\" (UID: \"82740182-3d7b-4d3c-8d49-8ed65c1ab143\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" Jul 21 12:46:32.890095 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.889874 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/82740182-3d7b-4d3c-8d49-8ed65c1ab143-webhook-cert\") pod \"opendatahub-operator-controller-manager-579d89bddd-pq59b\" (UID: \"82740182-3d7b-4d3c-8d49-8ed65c1ab143\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" Jul 21 12:46:32.890095 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.890007 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/82740182-3d7b-4d3c-8d49-8ed65c1ab143-apiservice-cert\") pod \"opendatahub-operator-controller-manager-579d89bddd-pq59b\" (UID: \"82740182-3d7b-4d3c-8d49-8ed65c1ab143\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" Jul 21 12:46:32.890095 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.890041 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69pxf\" (UniqueName: \"kubernetes.io/projected/82740182-3d7b-4d3c-8d49-8ed65c1ab143-kube-api-access-69pxf\") pod \"opendatahub-operator-controller-manager-579d89bddd-pq59b\" (UID: \"82740182-3d7b-4d3c-8d49-8ed65c1ab143\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" Jul 21 12:46:32.890374 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.890347 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/82740182-3d7b-4d3c-8d49-8ed65c1ab143-manifests\") pod \"opendatahub-operator-controller-manager-579d89bddd-pq59b\" (UID: \"82740182-3d7b-4d3c-8d49-8ed65c1ab143\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" Jul 21 12:46:32.892302 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.892278 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82740182-3d7b-4d3c-8d49-8ed65c1ab143-tmp\") pod \"opendatahub-operator-controller-manager-579d89bddd-pq59b\" (UID: \"82740182-3d7b-4d3c-8d49-8ed65c1ab143\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" Jul 21 12:46:32.892559 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.892541 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/82740182-3d7b-4d3c-8d49-8ed65c1ab143-webhook-cert\") pod \"opendatahub-operator-controller-manager-579d89bddd-pq59b\" (UID: \"82740182-3d7b-4d3c-8d49-8ed65c1ab143\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" Jul 21 12:46:32.892626 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.892580 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/82740182-3d7b-4d3c-8d49-8ed65c1ab143-apiservice-cert\") pod \"opendatahub-operator-controller-manager-579d89bddd-pq59b\" (UID: \"82740182-3d7b-4d3c-8d49-8ed65c1ab143\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" Jul 21 12:46:32.904191 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:32.904162 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69pxf\" (UniqueName: \"kubernetes.io/projected/82740182-3d7b-4d3c-8d49-8ed65c1ab143-kube-api-access-69pxf\") pod \"opendatahub-operator-controller-manager-579d89bddd-pq59b\" (UID: \"82740182-3d7b-4d3c-8d49-8ed65c1ab143\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" Jul 21 12:46:33.061323 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:33.061220 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" Jul 21 12:46:33.207252 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:33.207226 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b"] Jul 21 12:46:33.210260 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:46:33.210233 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82740182_3d7b_4d3c_8d49_8ed65c1ab143.slice/crio-94e57de118c88a9c55641166b29035783967ab41f6da118b37b50acb157c6e99 WatchSource:0}: Error finding container 94e57de118c88a9c55641166b29035783967ab41f6da118b37b50acb157c6e99: Status 404 returned error can't find the container with id 94e57de118c88a9c55641166b29035783967ab41f6da118b37b50acb157c6e99 Jul 21 12:46:33.774488 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:33.774452 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" event={"ID":"82740182-3d7b-4d3c-8d49-8ed65c1ab143","Type":"ContainerStarted","Data":"94e57de118c88a9c55641166b29035783967ab41f6da118b37b50acb157c6e99"} Jul 21 12:46:36.735232 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:36.735204 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-84bfb648b5-sfv9t" Jul 21 12:46:36.793532 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:36.793497 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" event={"ID":"82740182-3d7b-4d3c-8d49-8ed65c1ab143","Type":"ContainerStarted","Data":"48c50af7b375b6e7139b985bca9f9181bf4c47d41ba5af862412d7b01f0ac839"} Jul 21 12:46:37.804550 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:37.804447 2555 generic.go:356] "Generic (PLEG): container finished" podID="82740182-3d7b-4d3c-8d49-8ed65c1ab143" containerID="48c50af7b375b6e7139b985bca9f9181bf4c47d41ba5af862412d7b01f0ac839" exitCode=0 Jul 21 12:46:37.804957 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:37.804549 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" event={"ID":"82740182-3d7b-4d3c-8d49-8ed65c1ab143","Type":"ContainerDied","Data":"48c50af7b375b6e7139b985bca9f9181bf4c47d41ba5af862412d7b01f0ac839"} Jul 21 12:46:38.811412 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:38.811375 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" event={"ID":"82740182-3d7b-4d3c-8d49-8ed65c1ab143","Type":"ContainerStarted","Data":"1af78003e48407187a73a7fd44fcaef26452fc8b614fbee545826665d0b596ed"} Jul 21 12:46:38.811861 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:38.811473 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" Jul 21 12:46:38.833639 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:38.833548 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" podStartSLOduration=2.414860298 podStartE2EDuration="6.833531027s" totalImagesPullingTime="3.325178856s" totalInitContainerRuntime="1.093491873s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:46:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:46:38.83235921 +0000 UTC m=+421.450652191" watchObservedRunningTime="2026-07-21 12:46:38.833531027 +0000 UTC m=+421.451823975" Jul 21 12:46:49.818024 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:49.817986 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-pq59b" Jul 21 12:46:59.918011 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:59.917927 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww"] Jul 21 12:46:59.921304 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:59.921278 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:46:59.924153 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:59.924117 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 21 12:46:59.924431 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:59.924412 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-q58rx\"" Jul 21 12:46:59.924679 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:59.924665 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 21 12:46:59.924865 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:59.924841 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 21 12:46:59.924994 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:59.924842 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 21 12:46:59.935876 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:46:59.935848 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww"] Jul 21 12:47:00.032172 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.032139 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/486642b7-385b-496b-9ee8-f924d76c0ba6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.032172 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.032175 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/486642b7-385b-496b-9ee8-f924d76c0ba6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.032398 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.032207 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/486642b7-385b-496b-9ee8-f924d76c0ba6-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.032398 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.032269 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znkch\" (UniqueName: \"kubernetes.io/projected/486642b7-385b-496b-9ee8-f924d76c0ba6-kube-api-access-znkch\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.032398 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.032325 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/486642b7-385b-496b-9ee8-f924d76c0ba6-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.032398 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.032352 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/486642b7-385b-496b-9ee8-f924d76c0ba6-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.032542 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.032405 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/486642b7-385b-496b-9ee8-f924d76c0ba6-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.133579 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.133536 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/486642b7-385b-496b-9ee8-f924d76c0ba6-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.133806 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.133608 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znkch\" (UniqueName: \"kubernetes.io/projected/486642b7-385b-496b-9ee8-f924d76c0ba6-kube-api-access-znkch\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.133806 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.133668 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/486642b7-385b-496b-9ee8-f924d76c0ba6-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.133806 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.133703 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/486642b7-385b-496b-9ee8-f924d76c0ba6-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.133806 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.133731 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/486642b7-385b-496b-9ee8-f924d76c0ba6-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.134030 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.133873 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/486642b7-385b-496b-9ee8-f924d76c0ba6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.134162 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.134137 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/486642b7-385b-496b-9ee8-f924d76c0ba6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.135210 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.135178 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/486642b7-385b-496b-9ee8-f924d76c0ba6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.136608 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.136568 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/486642b7-385b-496b-9ee8-f924d76c0ba6-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.136819 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.136798 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/486642b7-385b-496b-9ee8-f924d76c0ba6-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.136890 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.136865 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/486642b7-385b-496b-9ee8-f924d76c0ba6-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.136931 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.136876 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/486642b7-385b-496b-9ee8-f924d76c0ba6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.143223 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.143189 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/486642b7-385b-496b-9ee8-f924d76c0ba6-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.143634 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.143585 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znkch\" (UniqueName: \"kubernetes.io/projected/486642b7-385b-496b-9ee8-f924d76c0ba6-kube-api-access-znkch\") pod \"istiod-openshift-gateway-94bb8fbfd-wqhww\" (UID: \"486642b7-385b-496b-9ee8-f924d76c0ba6\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.232619 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.232494 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:00.417303 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.417278 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww"] Jul 21 12:47:00.419847 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:47:00.419806 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod486642b7_385b_496b_9ee8_f924d76c0ba6.slice/crio-0e21df26c39c7d0f790d53dc998a98b1b4288633d283dc72b2d01f92301ac03c WatchSource:0}: Error finding container 0e21df26c39c7d0f790d53dc998a98b1b4288633d283dc72b2d01f92301ac03c: Status 404 returned error can't find the container with id 0e21df26c39c7d0f790d53dc998a98b1b4288633d283dc72b2d01f92301ac03c Jul 21 12:47:00.912936 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:00.912901 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" event={"ID":"486642b7-385b-496b-9ee8-f924d76c0ba6","Type":"ContainerStarted","Data":"0e21df26c39c7d0f790d53dc998a98b1b4288633d283dc72b2d01f92301ac03c"} Jul 21 12:47:02.823529 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:02.823488 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 12:47:02.823864 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:02.823562 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 12:47:02.926206 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:02.926168 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" event={"ID":"486642b7-385b-496b-9ee8-f924d76c0ba6","Type":"ContainerStarted","Data":"ef8091def1c08eecdd5da60e2ca7020bde703367c625d9b06a8f3560ea3dda6f"} Jul 21 12:47:02.926416 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:02.926320 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:47:02.968709 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:02.968654 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" podStartSLOduration=1.567411289 podStartE2EDuration="3.968635573s" totalImagesPullingTime="2.401224284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:47:02.965941607 +0000 UTC m=+445.584234555" watchObservedRunningTime="2026-07-21 12:47:02.968635573 +0000 UTC m=+445.586928517" Jul 21 12:47:03.932437 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:47:03.932407 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wqhww" Jul 21 12:48:08.748302 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:08.748267 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf"] Jul 21 12:48:08.751840 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:08.751819 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf" Jul 21 12:48:08.754730 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:08.754712 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 21 12:48:08.755135 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:08.755117 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 21 12:48:08.757362 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:08.757334 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hsjs2\"" Jul 21 12:48:08.765052 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:08.765024 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf"] Jul 21 12:48:08.826810 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:08.826770 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/952c588a-0e50-48c7-8460-b5bfba7189d8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf\" (UID: \"952c588a-0e50-48c7-8460-b5bfba7189d8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf" Jul 21 12:48:08.826810 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:08.826810 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/952c588a-0e50-48c7-8460-b5bfba7189d8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf\" (UID: \"952c588a-0e50-48c7-8460-b5bfba7189d8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf" Jul 21 12:48:08.827054 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:08.826833 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2s9c\" (UniqueName: \"kubernetes.io/projected/952c588a-0e50-48c7-8460-b5bfba7189d8-kube-api-access-r2s9c\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf\" (UID: \"952c588a-0e50-48c7-8460-b5bfba7189d8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf" Jul 21 12:48:08.928356 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:08.928314 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/952c588a-0e50-48c7-8460-b5bfba7189d8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf\" (UID: \"952c588a-0e50-48c7-8460-b5bfba7189d8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf" Jul 21 12:48:08.928356 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:08.928355 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/952c588a-0e50-48c7-8460-b5bfba7189d8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf\" (UID: \"952c588a-0e50-48c7-8460-b5bfba7189d8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf" Jul 21 12:48:08.928565 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:08.928383 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2s9c\" (UniqueName: \"kubernetes.io/projected/952c588a-0e50-48c7-8460-b5bfba7189d8-kube-api-access-r2s9c\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf\" (UID: \"952c588a-0e50-48c7-8460-b5bfba7189d8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf" Jul 21 12:48:08.928746 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:08.928728 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/952c588a-0e50-48c7-8460-b5bfba7189d8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf\" (UID: \"952c588a-0e50-48c7-8460-b5bfba7189d8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf" Jul 21 12:48:08.928786 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:08.928766 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/952c588a-0e50-48c7-8460-b5bfba7189d8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf\" (UID: \"952c588a-0e50-48c7-8460-b5bfba7189d8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf" Jul 21 12:48:08.939134 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:08.939094 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2s9c\" (UniqueName: \"kubernetes.io/projected/952c588a-0e50-48c7-8460-b5bfba7189d8-kube-api-access-r2s9c\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf\" (UID: \"952c588a-0e50-48c7-8460-b5bfba7189d8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf" Jul 21 12:48:09.061336 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.061232 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf" Jul 21 12:48:09.211738 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.211700 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf"] Jul 21 12:48:09.215326 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:48:09.215288 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod952c588a_0e50_48c7_8460_b5bfba7189d8.slice/crio-69e0df33fb3bfb7c83f3c73e3667b06f5b2f95f09e330817217cc4be03283bd1 WatchSource:0}: Error finding container 69e0df33fb3bfb7c83f3c73e3667b06f5b2f95f09e330817217cc4be03283bd1: Status 404 returned error can't find the container with id 69e0df33fb3bfb7c83f3c73e3667b06f5b2f95f09e330817217cc4be03283bd1 Jul 21 12:48:09.242028 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.241999 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf" event={"ID":"952c588a-0e50-48c7-8460-b5bfba7189d8","Type":"ContainerStarted","Data":"69e0df33fb3bfb7c83f3c73e3667b06f5b2f95f09e330817217cc4be03283bd1"} Jul 21 12:48:09.326620 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.326569 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk"] Jul 21 12:48:09.330283 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.330259 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk" Jul 21 12:48:09.341113 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.341080 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk"] Jul 21 12:48:09.433263 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.433214 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9hlb\" (UniqueName: \"kubernetes.io/projected/3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf-kube-api-access-v9hlb\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk\" (UID: \"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk" Jul 21 12:48:09.433455 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.433271 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk\" (UID: \"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk" Jul 21 12:48:09.433455 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.433343 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk\" (UID: \"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk" Jul 21 12:48:09.534691 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.534650 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9hlb\" (UniqueName: \"kubernetes.io/projected/3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf-kube-api-access-v9hlb\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk\" (UID: \"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk" Jul 21 12:48:09.534691 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.534701 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk\" (UID: \"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk" Jul 21 12:48:09.534907 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.534736 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk\" (UID: \"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk" Jul 21 12:48:09.535109 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.535091 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk\" (UID: \"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk" Jul 21 12:48:09.535147 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.535124 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk\" (UID: \"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk" Jul 21 12:48:09.545717 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.545684 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9hlb\" (UniqueName: \"kubernetes.io/projected/3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf-kube-api-access-v9hlb\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk\" (UID: \"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk" Jul 21 12:48:09.679578 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.679539 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk" Jul 21 12:48:09.741922 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.741894 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl"] Jul 21 12:48:09.745534 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.745512 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl" Jul 21 12:48:09.755858 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.755771 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl"] Jul 21 12:48:09.826754 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.826704 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk"] Jul 21 12:48:09.830965 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:48:09.830933 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cbaf6ab_5fa5_416a_ba27_fc99281d2aaf.slice/crio-633c327e54a465dc88f995a2652d3127b5bf96f28a9d43d1919736b4e1482ddc WatchSource:0}: Error finding container 633c327e54a465dc88f995a2652d3127b5bf96f28a9d43d1919736b4e1482ddc: Status 404 returned error can't find the container with id 633c327e54a465dc88f995a2652d3127b5bf96f28a9d43d1919736b4e1482ddc Jul 21 12:48:09.837906 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.837878 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/caf82597-ec5e-4e4c-8546-2123c195bcbd-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl\" (UID: \"caf82597-ec5e-4e4c-8546-2123c195bcbd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl" Jul 21 12:48:09.837906 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.837913 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz5mw\" (UniqueName: \"kubernetes.io/projected/caf82597-ec5e-4e4c-8546-2123c195bcbd-kube-api-access-qz5mw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl\" (UID: \"caf82597-ec5e-4e4c-8546-2123c195bcbd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl" Jul 21 12:48:09.838077 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.837934 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/caf82597-ec5e-4e4c-8546-2123c195bcbd-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl\" (UID: \"caf82597-ec5e-4e4c-8546-2123c195bcbd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl" Jul 21 12:48:09.938747 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.938714 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/caf82597-ec5e-4e4c-8546-2123c195bcbd-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl\" (UID: \"caf82597-ec5e-4e4c-8546-2123c195bcbd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl" Jul 21 12:48:09.938966 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.938755 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz5mw\" (UniqueName: \"kubernetes.io/projected/caf82597-ec5e-4e4c-8546-2123c195bcbd-kube-api-access-qz5mw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl\" (UID: \"caf82597-ec5e-4e4c-8546-2123c195bcbd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl" Jul 21 12:48:09.938966 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.938785 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/caf82597-ec5e-4e4c-8546-2123c195bcbd-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl\" (UID: \"caf82597-ec5e-4e4c-8546-2123c195bcbd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl" Jul 21 12:48:09.939104 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.939067 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/caf82597-ec5e-4e4c-8546-2123c195bcbd-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl\" (UID: \"caf82597-ec5e-4e4c-8546-2123c195bcbd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl" Jul 21 12:48:09.939220 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.939200 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/caf82597-ec5e-4e4c-8546-2123c195bcbd-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl\" (UID: \"caf82597-ec5e-4e4c-8546-2123c195bcbd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl" Jul 21 12:48:09.949707 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:09.949677 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz5mw\" (UniqueName: \"kubernetes.io/projected/caf82597-ec5e-4e4c-8546-2123c195bcbd-kube-api-access-qz5mw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl\" (UID: \"caf82597-ec5e-4e4c-8546-2123c195bcbd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl" Jul 21 12:48:10.056069 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.056032 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl" Jul 21 12:48:10.138624 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.138567 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr"] Jul 21 12:48:10.143449 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.143419 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr" Jul 21 12:48:10.153504 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.153035 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr"] Jul 21 12:48:10.207633 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.207522 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl"] Jul 21 12:48:10.210501 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:48:10.210468 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaf82597_ec5e_4e4c_8546_2123c195bcbd.slice/crio-c1b38d1a9689fa32989472165587ed97938ff60279b6389e662a0c152c23c2ed WatchSource:0}: Error finding container c1b38d1a9689fa32989472165587ed97938ff60279b6389e662a0c152c23c2ed: Status 404 returned error can't find the container with id c1b38d1a9689fa32989472165587ed97938ff60279b6389e662a0c152c23c2ed Jul 21 12:48:10.241716 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.241676 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkntr\" (UniqueName: \"kubernetes.io/projected/60005c66-98c1-4cc0-8018-20461298604e-kube-api-access-dkntr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr\" (UID: \"60005c66-98c1-4cc0-8018-20461298604e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr" Jul 21 12:48:10.241909 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.241727 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60005c66-98c1-4cc0-8018-20461298604e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr\" (UID: \"60005c66-98c1-4cc0-8018-20461298604e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr" Jul 21 12:48:10.241909 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.241765 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60005c66-98c1-4cc0-8018-20461298604e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr\" (UID: \"60005c66-98c1-4cc0-8018-20461298604e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr" Jul 21 12:48:10.249179 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.249150 2555 generic.go:356] "Generic (PLEG): container finished" podID="952c588a-0e50-48c7-8460-b5bfba7189d8" containerID="21e45284b989cb89bde3fdf72dce195f73e50e0f110c4b0ed46e3baafb01ddf7" exitCode=0 Jul 21 12:48:10.249328 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.249245 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf" event={"ID":"952c588a-0e50-48c7-8460-b5bfba7189d8","Type":"ContainerDied","Data":"21e45284b989cb89bde3fdf72dce195f73e50e0f110c4b0ed46e3baafb01ddf7"} Jul 21 12:48:10.250609 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.250564 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl" event={"ID":"caf82597-ec5e-4e4c-8546-2123c195bcbd","Type":"ContainerStarted","Data":"c1b38d1a9689fa32989472165587ed97938ff60279b6389e662a0c152c23c2ed"} Jul 21 12:48:10.252277 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.252251 2555 generic.go:356] "Generic (PLEG): container finished" podID="3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf" containerID="74f2199e17e2e2902a6eb32b26037b96ba3940a0a81f165780570b7b7b41c656" exitCode=0 Jul 21 12:48:10.252401 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.252306 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk" event={"ID":"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf","Type":"ContainerDied","Data":"74f2199e17e2e2902a6eb32b26037b96ba3940a0a81f165780570b7b7b41c656"} Jul 21 12:48:10.252401 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.252380 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk" event={"ID":"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf","Type":"ContainerStarted","Data":"633c327e54a465dc88f995a2652d3127b5bf96f28a9d43d1919736b4e1482ddc"} Jul 21 12:48:10.342376 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.342339 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkntr\" (UniqueName: \"kubernetes.io/projected/60005c66-98c1-4cc0-8018-20461298604e-kube-api-access-dkntr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr\" (UID: \"60005c66-98c1-4cc0-8018-20461298604e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr" Jul 21 12:48:10.342524 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.342396 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60005c66-98c1-4cc0-8018-20461298604e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr\" (UID: \"60005c66-98c1-4cc0-8018-20461298604e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr" Jul 21 12:48:10.342524 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.342464 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60005c66-98c1-4cc0-8018-20461298604e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr\" (UID: \"60005c66-98c1-4cc0-8018-20461298604e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr" Jul 21 12:48:10.342825 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.342809 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60005c66-98c1-4cc0-8018-20461298604e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr\" (UID: \"60005c66-98c1-4cc0-8018-20461298604e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr" Jul 21 12:48:10.342873 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.342835 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60005c66-98c1-4cc0-8018-20461298604e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr\" (UID: \"60005c66-98c1-4cc0-8018-20461298604e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr" Jul 21 12:48:10.352810 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.352787 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkntr\" (UniqueName: \"kubernetes.io/projected/60005c66-98c1-4cc0-8018-20461298604e-kube-api-access-dkntr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr\" (UID: \"60005c66-98c1-4cc0-8018-20461298604e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr" Jul 21 12:48:10.458216 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.458127 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr" Jul 21 12:48:10.610585 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:10.610555 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr"] Jul 21 12:48:10.611081 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:48:10.611056 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60005c66_98c1_4cc0_8018_20461298604e.slice/crio-942dbd8bed82e409a180aa68865a21b7b6da8be916931d332de7e375f101c8da WatchSource:0}: Error finding container 942dbd8bed82e409a180aa68865a21b7b6da8be916931d332de7e375f101c8da: Status 404 returned error can't find the container with id 942dbd8bed82e409a180aa68865a21b7b6da8be916931d332de7e375f101c8da Jul 21 12:48:11.258985 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:11.258954 2555 generic.go:356] "Generic (PLEG): container finished" podID="caf82597-ec5e-4e4c-8546-2123c195bcbd" containerID="b9a7927d3e4566c0251b83cabf1a19d747541c0f2b27cd9cc42db3f5fe1fc8c0" exitCode=0 Jul 21 12:48:11.259428 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:11.259034 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl" event={"ID":"caf82597-ec5e-4e4c-8546-2123c195bcbd","Type":"ContainerDied","Data":"b9a7927d3e4566c0251b83cabf1a19d747541c0f2b27cd9cc42db3f5fe1fc8c0"} Jul 21 12:48:11.261010 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:11.260984 2555 generic.go:356] "Generic (PLEG): container finished" podID="3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf" containerID="bf8023a4b817596928b9d7cdbefaccdf8c20f00aede8c88025e06c7f2c585ca1" exitCode=0 Jul 21 12:48:11.261125 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:11.261055 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk" event={"ID":"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf","Type":"ContainerDied","Data":"bf8023a4b817596928b9d7cdbefaccdf8c20f00aede8c88025e06c7f2c585ca1"} Jul 21 12:48:11.263067 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:11.263046 2555 generic.go:356] "Generic (PLEG): container finished" podID="952c588a-0e50-48c7-8460-b5bfba7189d8" containerID="317c2fe5b98adbf8cd9350e8d65d29be52ed723a458fc583040f56242d9d1807" exitCode=0 Jul 21 12:48:11.263171 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:11.263115 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf" event={"ID":"952c588a-0e50-48c7-8460-b5bfba7189d8","Type":"ContainerDied","Data":"317c2fe5b98adbf8cd9350e8d65d29be52ed723a458fc583040f56242d9d1807"} Jul 21 12:48:11.264744 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:11.264723 2555 generic.go:356] "Generic (PLEG): container finished" podID="60005c66-98c1-4cc0-8018-20461298604e" containerID="b0cb2b1db3ce86877bcf8eb1b07fc2a1135af685acb29b38a50856f94bf55efe" exitCode=0 Jul 21 12:48:11.264820 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:11.264768 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr" event={"ID":"60005c66-98c1-4cc0-8018-20461298604e","Type":"ContainerDied","Data":"b0cb2b1db3ce86877bcf8eb1b07fc2a1135af685acb29b38a50856f94bf55efe"} Jul 21 12:48:11.264820 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:11.264806 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr" event={"ID":"60005c66-98c1-4cc0-8018-20461298604e","Type":"ContainerStarted","Data":"942dbd8bed82e409a180aa68865a21b7b6da8be916931d332de7e375f101c8da"} Jul 21 12:48:12.275779 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:12.275747 2555 generic.go:356] "Generic (PLEG): container finished" podID="952c588a-0e50-48c7-8460-b5bfba7189d8" containerID="54bb170a6d002160f785a233e24efb5d56dc3d2fe4c8d58bd92f2d5963d94ed1" exitCode=0 Jul 21 12:48:12.276220 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:12.275826 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf" event={"ID":"952c588a-0e50-48c7-8460-b5bfba7189d8","Type":"ContainerDied","Data":"54bb170a6d002160f785a233e24efb5d56dc3d2fe4c8d58bd92f2d5963d94ed1"} Jul 21 12:48:12.277720 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:12.277696 2555 generic.go:356] "Generic (PLEG): container finished" podID="3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf" containerID="5f154d8d8fe263e0a5139d6f81c55befc8e92d686b2e70d27f64f10bf8642b9d" exitCode=0 Jul 21 12:48:12.277848 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:12.277753 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk" event={"ID":"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf","Type":"ContainerDied","Data":"5f154d8d8fe263e0a5139d6f81c55befc8e92d686b2e70d27f64f10bf8642b9d"} Jul 21 12:48:13.285001 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.284966 2555 generic.go:356] "Generic (PLEG): container finished" podID="60005c66-98c1-4cc0-8018-20461298604e" containerID="b704ae58570390e6318da97bb6e009b48e363e99065f54c7c32f84eeec308b0b" exitCode=0 Jul 21 12:48:13.285448 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.285052 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr" event={"ID":"60005c66-98c1-4cc0-8018-20461298604e","Type":"ContainerDied","Data":"b704ae58570390e6318da97bb6e009b48e363e99065f54c7c32f84eeec308b0b"} Jul 21 12:48:13.287018 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.286992 2555 generic.go:356] "Generic (PLEG): container finished" podID="caf82597-ec5e-4e4c-8546-2123c195bcbd" containerID="cd352f67bb72d4eb590ea4220c0b93bd0013b3afcea277b6d5adeb9896ffac86" exitCode=0 Jul 21 12:48:13.287128 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.287021 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl" event={"ID":"caf82597-ec5e-4e4c-8546-2123c195bcbd","Type":"ContainerDied","Data":"cd352f67bb72d4eb590ea4220c0b93bd0013b3afcea277b6d5adeb9896ffac86"} Jul 21 12:48:13.450764 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.450739 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk" Jul 21 12:48:13.470374 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.470112 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf-util\") pod \"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf\" (UID: \"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf\") " Jul 21 12:48:13.470374 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.470215 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9hlb\" (UniqueName: \"kubernetes.io/projected/3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf-kube-api-access-v9hlb\") pod \"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf\" (UID: \"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf\") " Jul 21 12:48:13.470374 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.470284 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf-bundle\") pod \"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf\" (UID: \"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf\") " Jul 21 12:48:13.471581 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.471257 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf-bundle" (OuterVolumeSpecName: "bundle") pod "3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf" (UID: "3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:48:13.472919 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.472883 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf-kube-api-access-v9hlb" (OuterVolumeSpecName: "kube-api-access-v9hlb") pod "3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf" (UID: "3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf"). InnerVolumeSpecName "kube-api-access-v9hlb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:48:13.478095 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.478061 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf-util" (OuterVolumeSpecName: "util") pod "3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf" (UID: "3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:48:13.558834 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.558764 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf" Jul 21 12:48:13.570810 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.570786 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2s9c\" (UniqueName: \"kubernetes.io/projected/952c588a-0e50-48c7-8460-b5bfba7189d8-kube-api-access-r2s9c\") pod \"952c588a-0e50-48c7-8460-b5bfba7189d8\" (UID: \"952c588a-0e50-48c7-8460-b5bfba7189d8\") " Jul 21 12:48:13.570928 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.570884 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/952c588a-0e50-48c7-8460-b5bfba7189d8-bundle\") pod \"952c588a-0e50-48c7-8460-b5bfba7189d8\" (UID: \"952c588a-0e50-48c7-8460-b5bfba7189d8\") " Jul 21 12:48:13.570928 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.570917 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/952c588a-0e50-48c7-8460-b5bfba7189d8-util\") pod \"952c588a-0e50-48c7-8460-b5bfba7189d8\" (UID: \"952c588a-0e50-48c7-8460-b5bfba7189d8\") " Jul 21 12:48:13.571086 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.571076 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf-util\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:48:13.571128 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.571090 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9hlb\" (UniqueName: \"kubernetes.io/projected/3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf-kube-api-access-v9hlb\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:48:13.571128 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.571100 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf-bundle\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:48:13.571420 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.571365 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/952c588a-0e50-48c7-8460-b5bfba7189d8-bundle" (OuterVolumeSpecName: "bundle") pod "952c588a-0e50-48c7-8460-b5bfba7189d8" (UID: "952c588a-0e50-48c7-8460-b5bfba7189d8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:48:13.573081 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.573060 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/952c588a-0e50-48c7-8460-b5bfba7189d8-kube-api-access-r2s9c" (OuterVolumeSpecName: "kube-api-access-r2s9c") pod "952c588a-0e50-48c7-8460-b5bfba7189d8" (UID: "952c588a-0e50-48c7-8460-b5bfba7189d8"). InnerVolumeSpecName "kube-api-access-r2s9c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:48:13.576653 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.576626 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/952c588a-0e50-48c7-8460-b5bfba7189d8-util" (OuterVolumeSpecName: "util") pod "952c588a-0e50-48c7-8460-b5bfba7189d8" (UID: "952c588a-0e50-48c7-8460-b5bfba7189d8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:48:13.672275 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.672236 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/952c588a-0e50-48c7-8460-b5bfba7189d8-bundle\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:48:13.672275 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.672264 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/952c588a-0e50-48c7-8460-b5bfba7189d8-util\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:48:13.672275 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:13.672274 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2s9c\" (UniqueName: \"kubernetes.io/projected/952c588a-0e50-48c7-8460-b5bfba7189d8-kube-api-access-r2s9c\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:48:14.293566 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:14.293482 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf" Jul 21 12:48:14.293566 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:14.293476 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf" event={"ID":"952c588a-0e50-48c7-8460-b5bfba7189d8","Type":"ContainerDied","Data":"69e0df33fb3bfb7c83f3c73e3667b06f5b2f95f09e330817217cc4be03283bd1"} Jul 21 12:48:14.294188 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:14.293614 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69e0df33fb3bfb7c83f3c73e3667b06f5b2f95f09e330817217cc4be03283bd1" Jul 21 12:48:14.295700 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:14.295674 2555 generic.go:356] "Generic (PLEG): container finished" podID="60005c66-98c1-4cc0-8018-20461298604e" containerID="4e2f77578a6130ecb758c1794974b80b99c69c5d506b51035d785fad80ca563f" exitCode=0 Jul 21 12:48:14.295849 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:14.295791 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr" event={"ID":"60005c66-98c1-4cc0-8018-20461298604e","Type":"ContainerDied","Data":"4e2f77578a6130ecb758c1794974b80b99c69c5d506b51035d785fad80ca563f"} Jul 21 12:48:14.298018 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:14.297995 2555 generic.go:356] "Generic (PLEG): container finished" podID="caf82597-ec5e-4e4c-8546-2123c195bcbd" containerID="0dc5311912cb7ba508d924b4a5d645c96d7979902f49319a7a1aa774759900ea" exitCode=0 Jul 21 12:48:14.298121 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:14.298037 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl" event={"ID":"caf82597-ec5e-4e4c-8546-2123c195bcbd","Type":"ContainerDied","Data":"0dc5311912cb7ba508d924b4a5d645c96d7979902f49319a7a1aa774759900ea"} Jul 21 12:48:14.299842 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:14.299824 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk" Jul 21 12:48:14.299944 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:14.299850 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk" event={"ID":"3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf","Type":"ContainerDied","Data":"633c327e54a465dc88f995a2652d3127b5bf96f28a9d43d1919736b4e1482ddc"} Jul 21 12:48:14.299944 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:14.299879 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="633c327e54a465dc88f995a2652d3127b5bf96f28a9d43d1919736b4e1482ddc" Jul 21 12:48:15.451673 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.451644 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr" Jul 21 12:48:15.478863 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.478838 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl" Jul 21 12:48:15.488352 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.488321 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60005c66-98c1-4cc0-8018-20461298604e-util\") pod \"60005c66-98c1-4cc0-8018-20461298604e\" (UID: \"60005c66-98c1-4cc0-8018-20461298604e\") " Jul 21 12:48:15.488352 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.488351 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60005c66-98c1-4cc0-8018-20461298604e-bundle\") pod \"60005c66-98c1-4cc0-8018-20461298604e\" (UID: \"60005c66-98c1-4cc0-8018-20461298604e\") " Jul 21 12:48:15.488566 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.488435 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkntr\" (UniqueName: \"kubernetes.io/projected/60005c66-98c1-4cc0-8018-20461298604e-kube-api-access-dkntr\") pod \"60005c66-98c1-4cc0-8018-20461298604e\" (UID: \"60005c66-98c1-4cc0-8018-20461298604e\") " Jul 21 12:48:15.488881 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.488856 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60005c66-98c1-4cc0-8018-20461298604e-bundle" (OuterVolumeSpecName: "bundle") pod "60005c66-98c1-4cc0-8018-20461298604e" (UID: "60005c66-98c1-4cc0-8018-20461298604e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:48:15.490820 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.490793 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60005c66-98c1-4cc0-8018-20461298604e-kube-api-access-dkntr" (OuterVolumeSpecName: "kube-api-access-dkntr") pod "60005c66-98c1-4cc0-8018-20461298604e" (UID: "60005c66-98c1-4cc0-8018-20461298604e"). InnerVolumeSpecName "kube-api-access-dkntr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:48:15.497615 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.497541 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60005c66-98c1-4cc0-8018-20461298604e-util" (OuterVolumeSpecName: "util") pod "60005c66-98c1-4cc0-8018-20461298604e" (UID: "60005c66-98c1-4cc0-8018-20461298604e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:48:15.589242 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.589204 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qz5mw\" (UniqueName: \"kubernetes.io/projected/caf82597-ec5e-4e4c-8546-2123c195bcbd-kube-api-access-qz5mw\") pod \"caf82597-ec5e-4e4c-8546-2123c195bcbd\" (UID: \"caf82597-ec5e-4e4c-8546-2123c195bcbd\") " Jul 21 12:48:15.589437 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.589256 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/caf82597-ec5e-4e4c-8546-2123c195bcbd-bundle\") pod \"caf82597-ec5e-4e4c-8546-2123c195bcbd\" (UID: \"caf82597-ec5e-4e4c-8546-2123c195bcbd\") " Jul 21 12:48:15.589437 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.589380 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/caf82597-ec5e-4e4c-8546-2123c195bcbd-util\") pod \"caf82597-ec5e-4e4c-8546-2123c195bcbd\" (UID: \"caf82597-ec5e-4e4c-8546-2123c195bcbd\") " Jul 21 12:48:15.589656 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.589628 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkntr\" (UniqueName: \"kubernetes.io/projected/60005c66-98c1-4cc0-8018-20461298604e-kube-api-access-dkntr\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:48:15.589656 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.589646 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60005c66-98c1-4cc0-8018-20461298604e-util\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:48:15.589656 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.589656 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60005c66-98c1-4cc0-8018-20461298604e-bundle\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:48:15.589895 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.589822 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf82597-ec5e-4e4c-8546-2123c195bcbd-bundle" (OuterVolumeSpecName: "bundle") pod "caf82597-ec5e-4e4c-8546-2123c195bcbd" (UID: "caf82597-ec5e-4e4c-8546-2123c195bcbd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:48:15.591486 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.591463 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caf82597-ec5e-4e4c-8546-2123c195bcbd-kube-api-access-qz5mw" (OuterVolumeSpecName: "kube-api-access-qz5mw") pod "caf82597-ec5e-4e4c-8546-2123c195bcbd" (UID: "caf82597-ec5e-4e4c-8546-2123c195bcbd"). InnerVolumeSpecName "kube-api-access-qz5mw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:48:15.594563 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.594538 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf82597-ec5e-4e4c-8546-2123c195bcbd-util" (OuterVolumeSpecName: "util") pod "caf82597-ec5e-4e4c-8546-2123c195bcbd" (UID: "caf82597-ec5e-4e4c-8546-2123c195bcbd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 12:48:15.690187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.690141 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qz5mw\" (UniqueName: \"kubernetes.io/projected/caf82597-ec5e-4e4c-8546-2123c195bcbd-kube-api-access-qz5mw\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:48:15.690187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.690174 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/caf82597-ec5e-4e4c-8546-2123c195bcbd-bundle\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:48:15.690187 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:15.690183 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/caf82597-ec5e-4e4c-8546-2123c195bcbd-util\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:48:16.312473 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:16.312385 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr" Jul 21 12:48:16.312473 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:16.312390 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr" event={"ID":"60005c66-98c1-4cc0-8018-20461298604e","Type":"ContainerDied","Data":"942dbd8bed82e409a180aa68865a21b7b6da8be916931d332de7e375f101c8da"} Jul 21 12:48:16.312738 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:16.312516 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="942dbd8bed82e409a180aa68865a21b7b6da8be916931d332de7e375f101c8da" Jul 21 12:48:16.314662 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:16.314628 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl" event={"ID":"caf82597-ec5e-4e4c-8546-2123c195bcbd","Type":"ContainerDied","Data":"c1b38d1a9689fa32989472165587ed97938ff60279b6389e662a0c152c23c2ed"} Jul 21 12:48:16.314662 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:16.314651 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl" Jul 21 12:48:16.314863 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:16.314682 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c1b38d1a9689fa32989472165587ed97938ff60279b6389e662a0c152c23c2ed" Jul 21 12:48:32.675994 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:32.675915 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66c4d7cd5b-g46h4"] Jul 21 12:48:34.589607 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.589563 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-g5hcm"] Jul 21 12:48:34.590077 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.589994 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60005c66-98c1-4cc0-8018-20461298604e" containerName="pull" Jul 21 12:48:34.590077 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590009 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="60005c66-98c1-4cc0-8018-20461298604e" containerName="pull" Jul 21 12:48:34.590077 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590019 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="952c588a-0e50-48c7-8460-b5bfba7189d8" containerName="pull" Jul 21 12:48:34.590077 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590024 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="952c588a-0e50-48c7-8460-b5bfba7189d8" containerName="pull" Jul 21 12:48:34.590077 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590033 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="952c588a-0e50-48c7-8460-b5bfba7189d8" containerName="util" Jul 21 12:48:34.590077 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590039 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="952c588a-0e50-48c7-8460-b5bfba7189d8" containerName="util" Jul 21 12:48:34.590077 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590053 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="952c588a-0e50-48c7-8460-b5bfba7189d8" containerName="extract" Jul 21 12:48:34.590077 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590058 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="952c588a-0e50-48c7-8460-b5bfba7189d8" containerName="extract" Jul 21 12:48:34.590077 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590066 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf" containerName="util" Jul 21 12:48:34.590077 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590070 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf" containerName="util" Jul 21 12:48:34.590077 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590075 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caf82597-ec5e-4e4c-8546-2123c195bcbd" containerName="extract" Jul 21 12:48:34.590077 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590080 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="caf82597-ec5e-4e4c-8546-2123c195bcbd" containerName="extract" Jul 21 12:48:34.590477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590086 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60005c66-98c1-4cc0-8018-20461298604e" containerName="extract" Jul 21 12:48:34.590477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590091 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="60005c66-98c1-4cc0-8018-20461298604e" containerName="extract" Jul 21 12:48:34.590477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590097 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caf82597-ec5e-4e4c-8546-2123c195bcbd" containerName="pull" Jul 21 12:48:34.590477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590101 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="caf82597-ec5e-4e4c-8546-2123c195bcbd" containerName="pull" Jul 21 12:48:34.590477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590108 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60005c66-98c1-4cc0-8018-20461298604e" containerName="util" Jul 21 12:48:34.590477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590113 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="60005c66-98c1-4cc0-8018-20461298604e" containerName="util" Jul 21 12:48:34.590477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590117 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf" containerName="pull" Jul 21 12:48:34.590477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590122 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf" containerName="pull" Jul 21 12:48:34.590477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590127 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caf82597-ec5e-4e4c-8546-2123c195bcbd" containerName="util" Jul 21 12:48:34.590477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590133 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="caf82597-ec5e-4e4c-8546-2123c195bcbd" containerName="util" Jul 21 12:48:34.590477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590140 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf" containerName="extract" Jul 21 12:48:34.590477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590145 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf" containerName="extract" Jul 21 12:48:34.590477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590199 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="60005c66-98c1-4cc0-8018-20461298604e" containerName="extract" Jul 21 12:48:34.590477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590206 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="952c588a-0e50-48c7-8460-b5bfba7189d8" containerName="extract" Jul 21 12:48:34.590477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590212 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="caf82597-ec5e-4e4c-8546-2123c195bcbd" containerName="extract" Jul 21 12:48:34.590477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.590220 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf" containerName="extract" Jul 21 12:48:34.593276 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.593260 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g5hcm" Jul 21 12:48:34.597231 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.597209 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 21 12:48:34.597408 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.597286 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-tjmkr\"" Jul 21 12:48:34.597408 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.597332 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 21 12:48:34.597787 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.597758 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 21 12:48:34.605669 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.605648 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-g5hcm"] Jul 21 12:48:34.654611 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.654568 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5rbs\" (UniqueName: \"kubernetes.io/projected/b481abe3-4169-4a16-9449-89ae4fcb3ba9-kube-api-access-x5rbs\") pod \"dns-operator-controller-manager-864dd78c4c-g5hcm\" (UID: \"b481abe3-4169-4a16-9449-89ae4fcb3ba9\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g5hcm" Jul 21 12:48:34.755310 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.755269 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5rbs\" (UniqueName: \"kubernetes.io/projected/b481abe3-4169-4a16-9449-89ae4fcb3ba9-kube-api-access-x5rbs\") pod \"dns-operator-controller-manager-864dd78c4c-g5hcm\" (UID: \"b481abe3-4169-4a16-9449-89ae4fcb3ba9\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g5hcm" Jul 21 12:48:34.765226 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.765195 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5rbs\" (UniqueName: \"kubernetes.io/projected/b481abe3-4169-4a16-9449-89ae4fcb3ba9-kube-api-access-x5rbs\") pod \"dns-operator-controller-manager-864dd78c4c-g5hcm\" (UID: \"b481abe3-4169-4a16-9449-89ae4fcb3ba9\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g5hcm" Jul 21 12:48:34.903211 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:34.903115 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g5hcm" Jul 21 12:48:35.055327 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:35.055293 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-g5hcm"] Jul 21 12:48:35.058290 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:48:35.058254 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb481abe3_4169_4a16_9449_89ae4fcb3ba9.slice/crio-cba05687d10c1c558a7cbe394f8c8920655a7af704017253fb548a1db4b54177 WatchSource:0}: Error finding container cba05687d10c1c558a7cbe394f8c8920655a7af704017253fb548a1db4b54177: Status 404 returned error can't find the container with id cba05687d10c1c558a7cbe394f8c8920655a7af704017253fb548a1db4b54177 Jul 21 12:48:35.412136 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:35.412100 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g5hcm" event={"ID":"b481abe3-4169-4a16-9449-89ae4fcb3ba9","Type":"ContainerStarted","Data":"cba05687d10c1c558a7cbe394f8c8920655a7af704017253fb548a1db4b54177"} Jul 21 12:48:37.426799 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:37.426763 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g5hcm" event={"ID":"b481abe3-4169-4a16-9449-89ae4fcb3ba9","Type":"ContainerStarted","Data":"01d3104a38962027c1599cc3d06b8d28f19f86f9ec0412abc89f93f31e12dae1"} Jul 21 12:48:37.427208 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:37.426895 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g5hcm" Jul 21 12:48:37.456623 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:37.456542 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g5hcm" podStartSLOduration=1.521527812 podStartE2EDuration="3.456524191s" totalImagesPullingTime="1.934996379s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:48:37.455068235 +0000 UTC m=+540.073361180" watchObservedRunningTime="2026-07-21 12:48:37.456524191 +0000 UTC m=+540.074817118" Jul 21 12:48:47.289212 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:47.289175 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq"] Jul 21 12:48:47.292544 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:47.292527 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq" Jul 21 12:48:47.296086 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:47.296055 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-622jg\"" Jul 21 12:48:47.329740 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:47.329705 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq"] Jul 21 12:48:47.466470 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:47.466429 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvw5k\" (UniqueName: \"kubernetes.io/projected/6e614cd5-6b44-4ea6-82fa-e45a6d59394b-kube-api-access-kvw5k\") pod \"kuadrant-operator-controller-manager-86878c769f-fqvkq\" (UID: \"6e614cd5-6b44-4ea6-82fa-e45a6d59394b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq" Jul 21 12:48:47.466679 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:47.466494 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e614cd5-6b44-4ea6-82fa-e45a6d59394b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fqvkq\" (UID: \"6e614cd5-6b44-4ea6-82fa-e45a6d59394b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq" Jul 21 12:48:47.567963 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:47.567863 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvw5k\" (UniqueName: \"kubernetes.io/projected/6e614cd5-6b44-4ea6-82fa-e45a6d59394b-kube-api-access-kvw5k\") pod \"kuadrant-operator-controller-manager-86878c769f-fqvkq\" (UID: \"6e614cd5-6b44-4ea6-82fa-e45a6d59394b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq" Jul 21 12:48:47.567963 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:47.567949 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e614cd5-6b44-4ea6-82fa-e45a6d59394b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fqvkq\" (UID: \"6e614cd5-6b44-4ea6-82fa-e45a6d59394b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq" Jul 21 12:48:47.568366 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:47.568337 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e614cd5-6b44-4ea6-82fa-e45a6d59394b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fqvkq\" (UID: \"6e614cd5-6b44-4ea6-82fa-e45a6d59394b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq" Jul 21 12:48:47.593915 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:47.593878 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvw5k\" (UniqueName: \"kubernetes.io/projected/6e614cd5-6b44-4ea6-82fa-e45a6d59394b-kube-api-access-kvw5k\") pod \"kuadrant-operator-controller-manager-86878c769f-fqvkq\" (UID: \"6e614cd5-6b44-4ea6-82fa-e45a6d59394b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq" Jul 21 12:48:47.603298 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:47.603268 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq" Jul 21 12:48:47.763878 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:47.763849 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq"] Jul 21 12:48:47.766694 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:48:47.766664 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e614cd5_6b44_4ea6_82fa_e45a6d59394b.slice/crio-cb977491ecee38c74c0649c8dffb1c0535881c66fd1878e6a7b91078b69bd1ba WatchSource:0}: Error finding container cb977491ecee38c74c0649c8dffb1c0535881c66fd1878e6a7b91078b69bd1ba: Status 404 returned error can't find the container with id cb977491ecee38c74c0649c8dffb1c0535881c66fd1878e6a7b91078b69bd1ba Jul 21 12:48:48.433333 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:48.433305 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-g5hcm" Jul 21 12:48:48.488456 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:48.488424 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq" event={"ID":"6e614cd5-6b44-4ea6-82fa-e45a6d59394b","Type":"ContainerStarted","Data":"cb977491ecee38c74c0649c8dffb1c0535881c66fd1878e6a7b91078b69bd1ba"} Jul 21 12:48:53.520467 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:53.520426 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq" event={"ID":"6e614cd5-6b44-4ea6-82fa-e45a6d59394b","Type":"ContainerStarted","Data":"f91b53a6e53cc5cd015cb57c4080efb013f28555e32d795dfdf230341cc41f52"} Jul 21 12:48:53.520942 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:53.520533 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq" Jul 21 12:48:53.558391 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:53.558341 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq" podStartSLOduration=1.827565671 podStartE2EDuration="6.558323295s" totalImagesPullingTime="4.730757624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:48:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:48:53.556422921 +0000 UTC m=+556.174715889" watchObservedRunningTime="2026-07-21 12:48:53.558323295 +0000 UTC m=+556.176616242" Jul 21 12:48:57.697578 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:57.697537 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66c4d7cd5b-g46h4" podUID="f7b03598-eacd-442b-b66d-88497e54683a" containerName="console" containerID="cri-o://a4af9ed236ade3960e829b730117e58845a70a5dde46ebe7a93a877a154e02af" gracePeriod=15 Jul 21 12:48:57.958808 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:57.958740 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:48:58.059417 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.059378 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9wqb\" (UniqueName: \"kubernetes.io/projected/f7b03598-eacd-442b-b66d-88497e54683a-kube-api-access-r9wqb\") pod \"f7b03598-eacd-442b-b66d-88497e54683a\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " Jul 21 12:48:58.059639 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.059437 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-trusted-ca-bundle\") pod \"f7b03598-eacd-442b-b66d-88497e54683a\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " Jul 21 12:48:58.059639 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.059459 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7b03598-eacd-442b-b66d-88497e54683a-console-serving-cert\") pod \"f7b03598-eacd-442b-b66d-88497e54683a\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " Jul 21 12:48:58.059639 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.059474 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-console-config\") pod \"f7b03598-eacd-442b-b66d-88497e54683a\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " Jul 21 12:48:58.059639 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.059496 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7b03598-eacd-442b-b66d-88497e54683a-console-oauth-config\") pod \"f7b03598-eacd-442b-b66d-88497e54683a\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " Jul 21 12:48:58.059639 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.059515 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-service-ca\") pod \"f7b03598-eacd-442b-b66d-88497e54683a\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " Jul 21 12:48:58.059639 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.059529 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-oauth-serving-cert\") pod \"f7b03598-eacd-442b-b66d-88497e54683a\" (UID: \"f7b03598-eacd-442b-b66d-88497e54683a\") " Jul 21 12:48:58.060025 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.059972 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f7b03598-eacd-442b-b66d-88497e54683a" (UID: "f7b03598-eacd-442b-b66d-88497e54683a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:48:58.060080 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.060017 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-console-config" (OuterVolumeSpecName: "console-config") pod "f7b03598-eacd-442b-b66d-88497e54683a" (UID: "f7b03598-eacd-442b-b66d-88497e54683a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:48:58.060080 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.060067 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f7b03598-eacd-442b-b66d-88497e54683a" (UID: "f7b03598-eacd-442b-b66d-88497e54683a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:48:58.060179 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.060160 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-service-ca" (OuterVolumeSpecName: "service-ca") pod "f7b03598-eacd-442b-b66d-88497e54683a" (UID: "f7b03598-eacd-442b-b66d-88497e54683a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 12:48:58.061962 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.061934 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7b03598-eacd-442b-b66d-88497e54683a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f7b03598-eacd-442b-b66d-88497e54683a" (UID: "f7b03598-eacd-442b-b66d-88497e54683a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:48:58.062065 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.062003 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7b03598-eacd-442b-b66d-88497e54683a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f7b03598-eacd-442b-b66d-88497e54683a" (UID: "f7b03598-eacd-442b-b66d-88497e54683a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:48:58.062065 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.062009 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7b03598-eacd-442b-b66d-88497e54683a-kube-api-access-r9wqb" (OuterVolumeSpecName: "kube-api-access-r9wqb") pod "f7b03598-eacd-442b-b66d-88497e54683a" (UID: "f7b03598-eacd-442b-b66d-88497e54683a"). InnerVolumeSpecName "kube-api-access-r9wqb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:48:58.160239 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.160201 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-console-config\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:48:58.160239 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.160230 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7b03598-eacd-442b-b66d-88497e54683a-console-oauth-config\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:48:58.160239 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.160241 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-service-ca\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:48:58.160460 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.160250 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-oauth-serving-cert\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:48:58.160460 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.160258 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9wqb\" (UniqueName: \"kubernetes.io/projected/f7b03598-eacd-442b-b66d-88497e54683a-kube-api-access-r9wqb\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:48:58.160460 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.160266 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7b03598-eacd-442b-b66d-88497e54683a-trusted-ca-bundle\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:48:58.160460 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.160274 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7b03598-eacd-442b-b66d-88497e54683a-console-serving-cert\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:48:58.545942 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.545909 2555 generic.go:356] "Generic (PLEG): container finished" podID="f7b03598-eacd-442b-b66d-88497e54683a" containerID="a4af9ed236ade3960e829b730117e58845a70a5dde46ebe7a93a877a154e02af" exitCode=0 Jul 21 12:48:58.546165 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.545968 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66c4d7cd5b-g46h4" Jul 21 12:48:58.546165 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.545999 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66c4d7cd5b-g46h4" event={"ID":"f7b03598-eacd-442b-b66d-88497e54683a","Type":"ContainerDied","Data":"a4af9ed236ade3960e829b730117e58845a70a5dde46ebe7a93a877a154e02af"} Jul 21 12:48:58.546165 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.546049 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66c4d7cd5b-g46h4" event={"ID":"f7b03598-eacd-442b-b66d-88497e54683a","Type":"ContainerDied","Data":"69fc9d81791a75ace13f493a2cd5349dd54d0e71ee2fe19663ccb4031a9bb5bd"} Jul 21 12:48:58.546165 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.546071 2555 scope.go:117] "RemoveContainer" containerID="a4af9ed236ade3960e829b730117e58845a70a5dde46ebe7a93a877a154e02af" Jul 21 12:48:58.555554 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.555535 2555 scope.go:117] "RemoveContainer" containerID="a4af9ed236ade3960e829b730117e58845a70a5dde46ebe7a93a877a154e02af" Jul 21 12:48:58.555884 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:48:58.555866 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4af9ed236ade3960e829b730117e58845a70a5dde46ebe7a93a877a154e02af\": container with ID starting with a4af9ed236ade3960e829b730117e58845a70a5dde46ebe7a93a877a154e02af not found: ID does not exist" containerID="a4af9ed236ade3960e829b730117e58845a70a5dde46ebe7a93a877a154e02af" Jul 21 12:48:58.555942 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.555894 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4af9ed236ade3960e829b730117e58845a70a5dde46ebe7a93a877a154e02af"} err="failed to get container status \"a4af9ed236ade3960e829b730117e58845a70a5dde46ebe7a93a877a154e02af\": rpc error: code = NotFound desc = could not find container \"a4af9ed236ade3960e829b730117e58845a70a5dde46ebe7a93a877a154e02af\": container with ID starting with a4af9ed236ade3960e829b730117e58845a70a5dde46ebe7a93a877a154e02af not found: ID does not exist" Jul 21 12:48:58.571372 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.571340 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66c4d7cd5b-g46h4"] Jul 21 12:48:58.578625 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:58.575538 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66c4d7cd5b-g46h4"] Jul 21 12:48:59.925446 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:48:59.925408 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7b03598-eacd-442b-b66d-88497e54683a" path="/var/lib/kubelet/pods/f7b03598-eacd-442b-b66d-88497e54683a/volumes" Jul 21 12:49:04.527094 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:49:04.527061 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq" Jul 21 12:49:37.826994 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:49:37.826964 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 12:49:37.829621 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:49:37.829579 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 12:49:37.832292 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:49:37.832273 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 12:49:37.834943 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:49:37.834919 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 12:50:03.040854 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.040776 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b4b5bbbcc-42mn5"] Jul 21 12:50:03.041315 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.041120 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7b03598-eacd-442b-b66d-88497e54683a" containerName="console" Jul 21 12:50:03.041315 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.041133 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7b03598-eacd-442b-b66d-88497e54683a" containerName="console" Jul 21 12:50:03.041315 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.041199 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7b03598-eacd-442b-b66d-88497e54683a" containerName="console" Jul 21 12:50:03.044177 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.044155 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b4b5bbbcc-42mn5" Jul 21 12:50:03.047525 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.047490 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 21 12:50:03.047698 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.047647 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7gb5d\"" Jul 21 12:50:03.047903 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.047884 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 21 12:50:03.060073 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.060042 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b4b5bbbcc-42mn5"] Jul 21 12:50:03.118128 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.118090 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aabe92c2-c6a8-4ff2-923f-5380d4e3ac10-webhook-cert\") pod \"maas-controller-b4b5bbbcc-42mn5\" (UID: \"aabe92c2-c6a8-4ff2-923f-5380d4e3ac10\") " pod="opendatahub/maas-controller-b4b5bbbcc-42mn5" Jul 21 12:50:03.118317 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.118135 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmqkq\" (UniqueName: \"kubernetes.io/projected/aabe92c2-c6a8-4ff2-923f-5380d4e3ac10-kube-api-access-qmqkq\") pod \"maas-controller-b4b5bbbcc-42mn5\" (UID: \"aabe92c2-c6a8-4ff2-923f-5380d4e3ac10\") " pod="opendatahub/maas-controller-b4b5bbbcc-42mn5" Jul 21 12:50:03.218886 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.218849 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aabe92c2-c6a8-4ff2-923f-5380d4e3ac10-webhook-cert\") pod \"maas-controller-b4b5bbbcc-42mn5\" (UID: \"aabe92c2-c6a8-4ff2-923f-5380d4e3ac10\") " pod="opendatahub/maas-controller-b4b5bbbcc-42mn5" Jul 21 12:50:03.219066 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.218896 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmqkq\" (UniqueName: \"kubernetes.io/projected/aabe92c2-c6a8-4ff2-923f-5380d4e3ac10-kube-api-access-qmqkq\") pod \"maas-controller-b4b5bbbcc-42mn5\" (UID: \"aabe92c2-c6a8-4ff2-923f-5380d4e3ac10\") " pod="opendatahub/maas-controller-b4b5bbbcc-42mn5" Jul 21 12:50:03.221511 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.221488 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aabe92c2-c6a8-4ff2-923f-5380d4e3ac10-webhook-cert\") pod \"maas-controller-b4b5bbbcc-42mn5\" (UID: \"aabe92c2-c6a8-4ff2-923f-5380d4e3ac10\") " pod="opendatahub/maas-controller-b4b5bbbcc-42mn5" Jul 21 12:50:03.230170 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.230145 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmqkq\" (UniqueName: \"kubernetes.io/projected/aabe92c2-c6a8-4ff2-923f-5380d4e3ac10-kube-api-access-qmqkq\") pod \"maas-controller-b4b5bbbcc-42mn5\" (UID: \"aabe92c2-c6a8-4ff2-923f-5380d4e3ac10\") " pod="opendatahub/maas-controller-b4b5bbbcc-42mn5" Jul 21 12:50:03.283491 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.283444 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bf95d5449-884d8"] Jul 21 12:50:03.286955 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.286936 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bf95d5449-884d8" Jul 21 12:50:03.297022 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.296989 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bf95d5449-884d8"] Jul 21 12:50:03.320292 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.320253 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6sqk\" (UniqueName: \"kubernetes.io/projected/31d3bd46-6ad3-44f6-8a5a-652659ca01b1-kube-api-access-w6sqk\") pod \"maas-controller-6bf95d5449-884d8\" (UID: \"31d3bd46-6ad3-44f6-8a5a-652659ca01b1\") " pod="opendatahub/maas-controller-6bf95d5449-884d8" Jul 21 12:50:03.320475 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.320310 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/31d3bd46-6ad3-44f6-8a5a-652659ca01b1-webhook-cert\") pod \"maas-controller-6bf95d5449-884d8\" (UID: \"31d3bd46-6ad3-44f6-8a5a-652659ca01b1\") " pod="opendatahub/maas-controller-6bf95d5449-884d8" Jul 21 12:50:03.355215 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.355179 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b4b5bbbcc-42mn5" Jul 21 12:50:03.421060 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.421028 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6sqk\" (UniqueName: \"kubernetes.io/projected/31d3bd46-6ad3-44f6-8a5a-652659ca01b1-kube-api-access-w6sqk\") pod \"maas-controller-6bf95d5449-884d8\" (UID: \"31d3bd46-6ad3-44f6-8a5a-652659ca01b1\") " pod="opendatahub/maas-controller-6bf95d5449-884d8" Jul 21 12:50:03.421241 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.421074 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/31d3bd46-6ad3-44f6-8a5a-652659ca01b1-webhook-cert\") pod \"maas-controller-6bf95d5449-884d8\" (UID: \"31d3bd46-6ad3-44f6-8a5a-652659ca01b1\") " pod="opendatahub/maas-controller-6bf95d5449-884d8" Jul 21 12:50:03.424139 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.424075 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/31d3bd46-6ad3-44f6-8a5a-652659ca01b1-webhook-cert\") pod \"maas-controller-6bf95d5449-884d8\" (UID: \"31d3bd46-6ad3-44f6-8a5a-652659ca01b1\") " pod="opendatahub/maas-controller-6bf95d5449-884d8" Jul 21 12:50:03.432713 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.432679 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6sqk\" (UniqueName: \"kubernetes.io/projected/31d3bd46-6ad3-44f6-8a5a-652659ca01b1-kube-api-access-w6sqk\") pod \"maas-controller-6bf95d5449-884d8\" (UID: \"31d3bd46-6ad3-44f6-8a5a-652659ca01b1\") " pod="opendatahub/maas-controller-6bf95d5449-884d8" Jul 21 12:50:03.500856 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.500822 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b4b5bbbcc-42mn5"] Jul 21 12:50:03.506649 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:50:03.505437 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaabe92c2_c6a8_4ff2_923f_5380d4e3ac10.slice/crio-6c1011cb244d8fbb76d86c691bb67a7c25df3b23f926b9bb144f291729382daa WatchSource:0}: Error finding container 6c1011cb244d8fbb76d86c691bb67a7c25df3b23f926b9bb144f291729382daa: Status 404 returned error can't find the container with id 6c1011cb244d8fbb76d86c691bb67a7c25df3b23f926b9bb144f291729382daa Jul 21 12:50:03.599119 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.599084 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bf95d5449-884d8" Jul 21 12:50:03.748056 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.748026 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bf95d5449-884d8"] Jul 21 12:50:03.749830 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:50:03.749795 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31d3bd46_6ad3_44f6_8a5a_652659ca01b1.slice/crio-fc977cfd07ceeb61fa92dbd5ea197e00d4d9d274fc9d78ae6290a230206b0b72 WatchSource:0}: Error finding container fc977cfd07ceeb61fa92dbd5ea197e00d4d9d274fc9d78ae6290a230206b0b72: Status 404 returned error can't find the container with id fc977cfd07ceeb61fa92dbd5ea197e00d4d9d274fc9d78ae6290a230206b0b72 Jul 21 12:50:03.887856 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.887750 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b4b5bbbcc-42mn5" event={"ID":"aabe92c2-c6a8-4ff2-923f-5380d4e3ac10","Type":"ContainerStarted","Data":"6c1011cb244d8fbb76d86c691bb67a7c25df3b23f926b9bb144f291729382daa"} Jul 21 12:50:03.889033 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:03.889005 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bf95d5449-884d8" event={"ID":"31d3bd46-6ad3-44f6-8a5a-652659ca01b1","Type":"ContainerStarted","Data":"fc977cfd07ceeb61fa92dbd5ea197e00d4d9d274fc9d78ae6290a230206b0b72"} Jul 21 12:50:06.910581 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:06.910525 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b4b5bbbcc-42mn5" event={"ID":"aabe92c2-c6a8-4ff2-923f-5380d4e3ac10","Type":"ContainerStarted","Data":"874389926e45451903465d508aa584a9dae6a51416c9494833838473dbfcf413"} Jul 21 12:50:06.911119 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:06.910622 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b4b5bbbcc-42mn5" Jul 21 12:50:06.912174 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:06.912150 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bf95d5449-884d8" event={"ID":"31d3bd46-6ad3-44f6-8a5a-652659ca01b1","Type":"ContainerStarted","Data":"a7a520cca0a3d7f74ddd4a439bda2051bb1fe6da120eb7eb7fec22057b271e2b"} Jul 21 12:50:06.912308 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:06.912238 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bf95d5449-884d8" Jul 21 12:50:06.930023 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:06.929961 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b4b5bbbcc-42mn5" podStartSLOduration=1.409458748 podStartE2EDuration="3.929938676s" totalImagesPullingTime="2.520479928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:50:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:50:06.92854626 +0000 UTC m=+629.546839207" watchObservedRunningTime="2026-07-21 12:50:06.929938676 +0000 UTC m=+629.548231625" Jul 21 12:50:06.948304 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:06.948246 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bf95d5449-884d8" podStartSLOduration=1.664513763 podStartE2EDuration="3.948226162s" totalImagesPullingTime="2.283712399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:50:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:50:06.94713494 +0000 UTC m=+629.565427900" watchObservedRunningTime="2026-07-21 12:50:06.948226162 +0000 UTC m=+629.566519109" Jul 21 12:50:17.931659 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:17.931622 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bf95d5449-884d8" Jul 21 12:50:17.932141 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:17.931680 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b4b5bbbcc-42mn5" Jul 21 12:50:18.011732 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:18.011695 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b4b5bbbcc-42mn5"] Jul 21 12:50:18.011996 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:18.011968 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-b4b5bbbcc-42mn5" podUID="aabe92c2-c6a8-4ff2-923f-5380d4e3ac10" containerName="manager" containerID="cri-o://874389926e45451903465d508aa584a9dae6a51416c9494833838473dbfcf413" gracePeriod=30 Jul 21 12:50:18.266861 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:18.266833 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b4b5bbbcc-42mn5" Jul 21 12:50:18.358780 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:18.358740 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aabe92c2-c6a8-4ff2-923f-5380d4e3ac10-webhook-cert\") pod \"aabe92c2-c6a8-4ff2-923f-5380d4e3ac10\" (UID: \"aabe92c2-c6a8-4ff2-923f-5380d4e3ac10\") " Jul 21 12:50:18.358981 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:18.358830 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmqkq\" (UniqueName: \"kubernetes.io/projected/aabe92c2-c6a8-4ff2-923f-5380d4e3ac10-kube-api-access-qmqkq\") pod \"aabe92c2-c6a8-4ff2-923f-5380d4e3ac10\" (UID: \"aabe92c2-c6a8-4ff2-923f-5380d4e3ac10\") " Jul 21 12:50:18.361164 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:18.361126 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aabe92c2-c6a8-4ff2-923f-5380d4e3ac10-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "aabe92c2-c6a8-4ff2-923f-5380d4e3ac10" (UID: "aabe92c2-c6a8-4ff2-923f-5380d4e3ac10"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:50:18.361164 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:18.361136 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aabe92c2-c6a8-4ff2-923f-5380d4e3ac10-kube-api-access-qmqkq" (OuterVolumeSpecName: "kube-api-access-qmqkq") pod "aabe92c2-c6a8-4ff2-923f-5380d4e3ac10" (UID: "aabe92c2-c6a8-4ff2-923f-5380d4e3ac10"). InnerVolumeSpecName "kube-api-access-qmqkq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:50:18.459399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:18.459302 2555 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aabe92c2-c6a8-4ff2-923f-5380d4e3ac10-webhook-cert\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:50:18.459399 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:18.459332 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmqkq\" (UniqueName: \"kubernetes.io/projected/aabe92c2-c6a8-4ff2-923f-5380d4e3ac10-kube-api-access-qmqkq\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:50:18.979326 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:18.979295 2555 generic.go:356] "Generic (PLEG): container finished" podID="aabe92c2-c6a8-4ff2-923f-5380d4e3ac10" containerID="874389926e45451903465d508aa584a9dae6a51416c9494833838473dbfcf413" exitCode=0 Jul 21 12:50:18.979772 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:18.979350 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b4b5bbbcc-42mn5" Jul 21 12:50:18.979772 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:18.979371 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b4b5bbbcc-42mn5" event={"ID":"aabe92c2-c6a8-4ff2-923f-5380d4e3ac10","Type":"ContainerDied","Data":"874389926e45451903465d508aa584a9dae6a51416c9494833838473dbfcf413"} Jul 21 12:50:18.979772 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:18.979414 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b4b5bbbcc-42mn5" event={"ID":"aabe92c2-c6a8-4ff2-923f-5380d4e3ac10","Type":"ContainerDied","Data":"6c1011cb244d8fbb76d86c691bb67a7c25df3b23f926b9bb144f291729382daa"} Jul 21 12:50:18.979772 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:18.979435 2555 scope.go:117] "RemoveContainer" containerID="874389926e45451903465d508aa584a9dae6a51416c9494833838473dbfcf413" Jul 21 12:50:18.988983 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:18.988965 2555 scope.go:117] "RemoveContainer" containerID="874389926e45451903465d508aa584a9dae6a51416c9494833838473dbfcf413" Jul 21 12:50:18.989269 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:50:18.989248 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"874389926e45451903465d508aa584a9dae6a51416c9494833838473dbfcf413\": container with ID starting with 874389926e45451903465d508aa584a9dae6a51416c9494833838473dbfcf413 not found: ID does not exist" containerID="874389926e45451903465d508aa584a9dae6a51416c9494833838473dbfcf413" Jul 21 12:50:18.989316 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:18.989283 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"874389926e45451903465d508aa584a9dae6a51416c9494833838473dbfcf413"} err="failed to get container status \"874389926e45451903465d508aa584a9dae6a51416c9494833838473dbfcf413\": rpc error: code = NotFound desc = could not find container \"874389926e45451903465d508aa584a9dae6a51416c9494833838473dbfcf413\": container with ID starting with 874389926e45451903465d508aa584a9dae6a51416c9494833838473dbfcf413 not found: ID does not exist" Jul 21 12:50:19.001612 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:19.001559 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b4b5bbbcc-42mn5"] Jul 21 12:50:19.004798 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:19.004774 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-b4b5bbbcc-42mn5"] Jul 21 12:50:19.925557 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:50:19.925522 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aabe92c2-c6a8-4ff2-923f-5380d4e3ac10" path="/var/lib/kubelet/pods/aabe92c2-c6a8-4ff2-923f-5380d4e3ac10/volumes" Jul 21 12:51:19.361504 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.361472 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh"] Jul 21 12:51:19.361944 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.361896 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aabe92c2-c6a8-4ff2-923f-5380d4e3ac10" containerName="manager" Jul 21 12:51:19.361944 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.361910 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="aabe92c2-c6a8-4ff2-923f-5380d4e3ac10" containerName="manager" Jul 21 12:51:19.362019 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.361980 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="aabe92c2-c6a8-4ff2-923f-5380d4e3ac10" containerName="manager" Jul 21 12:51:19.365088 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.365064 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.367579 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.367550 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 21 12:51:19.367826 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.367810 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 21 12:51:19.368323 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.368310 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 21 12:51:19.368628 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.368589 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gbts6\"" Jul 21 12:51:19.376536 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.376511 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh"] Jul 21 12:51:19.443730 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.443693 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84bd219d-205e-4c6c-8382-f007b8fc54da-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rjvxh\" (UID: \"84bd219d-205e-4c6c-8382-f007b8fc54da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.443901 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.443751 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84bd219d-205e-4c6c-8382-f007b8fc54da-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rjvxh\" (UID: \"84bd219d-205e-4c6c-8382-f007b8fc54da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.443901 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.443804 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84bd219d-205e-4c6c-8382-f007b8fc54da-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rjvxh\" (UID: \"84bd219d-205e-4c6c-8382-f007b8fc54da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.443901 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.443858 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2xqv\" (UniqueName: \"kubernetes.io/projected/84bd219d-205e-4c6c-8382-f007b8fc54da-kube-api-access-j2xqv\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rjvxh\" (UID: \"84bd219d-205e-4c6c-8382-f007b8fc54da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.443901 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.443896 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84bd219d-205e-4c6c-8382-f007b8fc54da-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rjvxh\" (UID: \"84bd219d-205e-4c6c-8382-f007b8fc54da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.444080 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.443950 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84bd219d-205e-4c6c-8382-f007b8fc54da-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rjvxh\" (UID: \"84bd219d-205e-4c6c-8382-f007b8fc54da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.544689 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.544650 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84bd219d-205e-4c6c-8382-f007b8fc54da-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rjvxh\" (UID: \"84bd219d-205e-4c6c-8382-f007b8fc54da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.544886 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.544714 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84bd219d-205e-4c6c-8382-f007b8fc54da-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rjvxh\" (UID: \"84bd219d-205e-4c6c-8382-f007b8fc54da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.544886 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.544747 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84bd219d-205e-4c6c-8382-f007b8fc54da-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rjvxh\" (UID: \"84bd219d-205e-4c6c-8382-f007b8fc54da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.544886 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.544774 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84bd219d-205e-4c6c-8382-f007b8fc54da-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rjvxh\" (UID: \"84bd219d-205e-4c6c-8382-f007b8fc54da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.544886 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.544821 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2xqv\" (UniqueName: \"kubernetes.io/projected/84bd219d-205e-4c6c-8382-f007b8fc54da-kube-api-access-j2xqv\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rjvxh\" (UID: \"84bd219d-205e-4c6c-8382-f007b8fc54da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.545463 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.544887 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84bd219d-205e-4c6c-8382-f007b8fc54da-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rjvxh\" (UID: \"84bd219d-205e-4c6c-8382-f007b8fc54da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.545463 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.545160 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84bd219d-205e-4c6c-8382-f007b8fc54da-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rjvxh\" (UID: \"84bd219d-205e-4c6c-8382-f007b8fc54da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.545463 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.545222 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84bd219d-205e-4c6c-8382-f007b8fc54da-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rjvxh\" (UID: \"84bd219d-205e-4c6c-8382-f007b8fc54da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.545463 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.545232 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84bd219d-205e-4c6c-8382-f007b8fc54da-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rjvxh\" (UID: \"84bd219d-205e-4c6c-8382-f007b8fc54da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.547190 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.547167 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84bd219d-205e-4c6c-8382-f007b8fc54da-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rjvxh\" (UID: \"84bd219d-205e-4c6c-8382-f007b8fc54da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.547496 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.547475 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84bd219d-205e-4c6c-8382-f007b8fc54da-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rjvxh\" (UID: \"84bd219d-205e-4c6c-8382-f007b8fc54da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.554004 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.553978 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2xqv\" (UniqueName: \"kubernetes.io/projected/84bd219d-205e-4c6c-8382-f007b8fc54da-kube-api-access-j2xqv\") pod \"e2e-trlp-test-simulated-kserve-598466548d-rjvxh\" (UID: \"84bd219d-205e-4c6c-8382-f007b8fc54da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.675855 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.675771 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:19.824424 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.817179 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh"] Jul 21 12:51:19.824424 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:19.822003 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 12:51:20.309963 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:20.309919 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" event={"ID":"84bd219d-205e-4c6c-8382-f007b8fc54da","Type":"ContainerStarted","Data":"83e0ce3921fef2e758bda7fc27b306e2ade8a60a944045adb9c4ecbba722d084"} Jul 21 12:51:22.323057 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:22.323021 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" event={"ID":"84bd219d-205e-4c6c-8382-f007b8fc54da","Type":"ContainerStarted","Data":"7df1ad4808928459869b7405b3ea9a95187228678f14af1b302de5ebc2f4b138"} Jul 21 12:51:22.346457 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:22.346401 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" podStartSLOduration=1.767029752 podStartE2EDuration="3.346382313s" totalImagesPullingTime="1.579352561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:22.346203738 +0000 UTC m=+704.964496686" watchObservedRunningTime="2026-07-21 12:51:22.346382313 +0000 UTC m=+704.964675260" Jul 21 12:51:24.562822 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.562783 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k"] Jul 21 12:51:24.566651 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.566629 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:24.568925 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.568901 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 21 12:51:24.577896 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.577866 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k"] Jul 21 12:51:24.592674 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.592635 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47702c39-f4bf-4057-963b-dadf58b60cd8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k\" (UID: \"47702c39-f4bf-4057-963b-dadf58b60cd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:24.592866 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.592698 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47702c39-f4bf-4057-963b-dadf58b60cd8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k\" (UID: \"47702c39-f4bf-4057-963b-dadf58b60cd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:24.592866 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.592745 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47702c39-f4bf-4057-963b-dadf58b60cd8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k\" (UID: \"47702c39-f4bf-4057-963b-dadf58b60cd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:24.592866 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.592796 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47702c39-f4bf-4057-963b-dadf58b60cd8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k\" (UID: \"47702c39-f4bf-4057-963b-dadf58b60cd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:24.592866 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.592826 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47702c39-f4bf-4057-963b-dadf58b60cd8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k\" (UID: \"47702c39-f4bf-4057-963b-dadf58b60cd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:24.592866 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.592854 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5qbg\" (UniqueName: \"kubernetes.io/projected/47702c39-f4bf-4057-963b-dadf58b60cd8-kube-api-access-v5qbg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k\" (UID: \"47702c39-f4bf-4057-963b-dadf58b60cd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:24.694275 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.694240 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47702c39-f4bf-4057-963b-dadf58b60cd8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k\" (UID: \"47702c39-f4bf-4057-963b-dadf58b60cd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:24.694443 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.694285 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47702c39-f4bf-4057-963b-dadf58b60cd8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k\" (UID: \"47702c39-f4bf-4057-963b-dadf58b60cd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:24.694443 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.694412 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47702c39-f4bf-4057-963b-dadf58b60cd8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k\" (UID: \"47702c39-f4bf-4057-963b-dadf58b60cd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:24.694512 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.694483 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47702c39-f4bf-4057-963b-dadf58b60cd8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k\" (UID: \"47702c39-f4bf-4057-963b-dadf58b60cd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:24.694545 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.694524 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47702c39-f4bf-4057-963b-dadf58b60cd8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k\" (UID: \"47702c39-f4bf-4057-963b-dadf58b60cd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:24.694612 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.694556 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5qbg\" (UniqueName: \"kubernetes.io/projected/47702c39-f4bf-4057-963b-dadf58b60cd8-kube-api-access-v5qbg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k\" (UID: \"47702c39-f4bf-4057-963b-dadf58b60cd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:24.694768 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.694746 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47702c39-f4bf-4057-963b-dadf58b60cd8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k\" (UID: \"47702c39-f4bf-4057-963b-dadf58b60cd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:24.694843 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.694752 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47702c39-f4bf-4057-963b-dadf58b60cd8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k\" (UID: \"47702c39-f4bf-4057-963b-dadf58b60cd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:24.694843 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.694806 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47702c39-f4bf-4057-963b-dadf58b60cd8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k\" (UID: \"47702c39-f4bf-4057-963b-dadf58b60cd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:24.697150 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.697110 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47702c39-f4bf-4057-963b-dadf58b60cd8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k\" (UID: \"47702c39-f4bf-4057-963b-dadf58b60cd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:24.697331 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.697309 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47702c39-f4bf-4057-963b-dadf58b60cd8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k\" (UID: \"47702c39-f4bf-4057-963b-dadf58b60cd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:24.703587 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.703558 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5qbg\" (UniqueName: \"kubernetes.io/projected/47702c39-f4bf-4057-963b-dadf58b60cd8-kube-api-access-v5qbg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k\" (UID: \"47702c39-f4bf-4057-963b-dadf58b60cd8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:24.879786 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:24.879728 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:25.036038 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:25.036008 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k"] Jul 21 12:51:25.037509 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:51:25.037468 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47702c39_f4bf_4057_963b_dadf58b60cd8.slice/crio-e83ccc21b889b191baca2a8db7217aa18f4bd75971b92f892fda97fa45cd3e5e WatchSource:0}: Error finding container e83ccc21b889b191baca2a8db7217aa18f4bd75971b92f892fda97fa45cd3e5e: Status 404 returned error can't find the container with id e83ccc21b889b191baca2a8db7217aa18f4bd75971b92f892fda97fa45cd3e5e Jul 21 12:51:25.341032 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:25.341001 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" event={"ID":"47702c39-f4bf-4057-963b-dadf58b60cd8","Type":"ContainerStarted","Data":"e83ccc21b889b191baca2a8db7217aa18f4bd75971b92f892fda97fa45cd3e5e"} Jul 21 12:51:26.347841 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:26.347805 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" event={"ID":"47702c39-f4bf-4057-963b-dadf58b60cd8","Type":"ContainerStarted","Data":"42644bdc418cd4e7a70566070a43d9e4f87b6d6ab1e80785f0d49d1abdcb65be"} Jul 21 12:51:26.368875 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:26.368823 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" podStartSLOduration=1.966221359 podStartE2EDuration="2.36880596s" totalImagesPullingTime="402.584601ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:51:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:26.367363179 +0000 UTC m=+708.985656128" watchObservedRunningTime="2026-07-21 12:51:26.36880596 +0000 UTC m=+708.987098908" Jul 21 12:51:29.676706 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:29.676586 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:29.676706 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:29.676686 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:29.689323 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:29.689295 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:30.379833 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:30.379803 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-rjvxh" Jul 21 12:51:34.880577 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:34.880538 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:34.881022 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:34.880621 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:34.893380 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:34.893346 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:51:35.405823 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:51:35.405792 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k" Jul 21 12:53:50.778200 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:53:50.778166 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bf95d5449-884d8"] Jul 21 12:53:50.778711 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:53:50.778366 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6bf95d5449-884d8" podUID="31d3bd46-6ad3-44f6-8a5a-652659ca01b1" containerName="manager" containerID="cri-o://a7a520cca0a3d7f74ddd4a439bda2051bb1fe6da120eb7eb7fec22057b271e2b" gracePeriod=30 Jul 21 12:53:51.033944 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:53:51.033885 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bf95d5449-884d8" Jul 21 12:53:51.106284 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:53:51.106244 2555 generic.go:356] "Generic (PLEG): container finished" podID="31d3bd46-6ad3-44f6-8a5a-652659ca01b1" containerID="a7a520cca0a3d7f74ddd4a439bda2051bb1fe6da120eb7eb7fec22057b271e2b" exitCode=0 Jul 21 12:53:51.106474 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:53:51.106302 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bf95d5449-884d8" Jul 21 12:53:51.106474 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:53:51.106332 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bf95d5449-884d8" event={"ID":"31d3bd46-6ad3-44f6-8a5a-652659ca01b1","Type":"ContainerDied","Data":"a7a520cca0a3d7f74ddd4a439bda2051bb1fe6da120eb7eb7fec22057b271e2b"} Jul 21 12:53:51.106474 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:53:51.106393 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bf95d5449-884d8" event={"ID":"31d3bd46-6ad3-44f6-8a5a-652659ca01b1","Type":"ContainerDied","Data":"fc977cfd07ceeb61fa92dbd5ea197e00d4d9d274fc9d78ae6290a230206b0b72"} Jul 21 12:53:51.106474 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:53:51.106419 2555 scope.go:117] "RemoveContainer" containerID="a7a520cca0a3d7f74ddd4a439bda2051bb1fe6da120eb7eb7fec22057b271e2b" Jul 21 12:53:51.118084 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:53:51.118063 2555 scope.go:117] "RemoveContainer" containerID="a7a520cca0a3d7f74ddd4a439bda2051bb1fe6da120eb7eb7fec22057b271e2b" Jul 21 12:53:51.118363 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:53:51.118344 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7a520cca0a3d7f74ddd4a439bda2051bb1fe6da120eb7eb7fec22057b271e2b\": container with ID starting with a7a520cca0a3d7f74ddd4a439bda2051bb1fe6da120eb7eb7fec22057b271e2b not found: ID does not exist" containerID="a7a520cca0a3d7f74ddd4a439bda2051bb1fe6da120eb7eb7fec22057b271e2b" Jul 21 12:53:51.118409 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:53:51.118374 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7a520cca0a3d7f74ddd4a439bda2051bb1fe6da120eb7eb7fec22057b271e2b"} err="failed to get container status \"a7a520cca0a3d7f74ddd4a439bda2051bb1fe6da120eb7eb7fec22057b271e2b\": rpc error: code = NotFound desc = could not find container \"a7a520cca0a3d7f74ddd4a439bda2051bb1fe6da120eb7eb7fec22057b271e2b\": container with ID starting with a7a520cca0a3d7f74ddd4a439bda2051bb1fe6da120eb7eb7fec22057b271e2b not found: ID does not exist" Jul 21 12:53:51.165224 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:53:51.165185 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6sqk\" (UniqueName: \"kubernetes.io/projected/31d3bd46-6ad3-44f6-8a5a-652659ca01b1-kube-api-access-w6sqk\") pod \"31d3bd46-6ad3-44f6-8a5a-652659ca01b1\" (UID: \"31d3bd46-6ad3-44f6-8a5a-652659ca01b1\") " Jul 21 12:53:51.165224 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:53:51.165234 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/31d3bd46-6ad3-44f6-8a5a-652659ca01b1-webhook-cert\") pod \"31d3bd46-6ad3-44f6-8a5a-652659ca01b1\" (UID: \"31d3bd46-6ad3-44f6-8a5a-652659ca01b1\") " Jul 21 12:53:51.167633 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:53:51.167577 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31d3bd46-6ad3-44f6-8a5a-652659ca01b1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "31d3bd46-6ad3-44f6-8a5a-652659ca01b1" (UID: "31d3bd46-6ad3-44f6-8a5a-652659ca01b1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:53:51.167633 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:53:51.167611 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31d3bd46-6ad3-44f6-8a5a-652659ca01b1-kube-api-access-w6sqk" (OuterVolumeSpecName: "kube-api-access-w6sqk") pod "31d3bd46-6ad3-44f6-8a5a-652659ca01b1" (UID: "31d3bd46-6ad3-44f6-8a5a-652659ca01b1"). InnerVolumeSpecName "kube-api-access-w6sqk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:53:51.266449 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:53:51.266409 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6sqk\" (UniqueName: \"kubernetes.io/projected/31d3bd46-6ad3-44f6-8a5a-652659ca01b1-kube-api-access-w6sqk\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:53:51.266449 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:53:51.266443 2555 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/31d3bd46-6ad3-44f6-8a5a-652659ca01b1-webhook-cert\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:53:51.433122 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:53:51.433076 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bf95d5449-884d8"] Jul 21 12:53:51.435732 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:53:51.435681 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6bf95d5449-884d8"] Jul 21 12:53:51.929662 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:53:51.929628 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31d3bd46-6ad3-44f6-8a5a-652659ca01b1" path="/var/lib/kubelet/pods/31d3bd46-6ad3-44f6-8a5a-652659ca01b1/volumes" Jul 21 12:54:37.864569 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:37.864496 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 12:54:37.865507 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:37.865488 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 12:54:37.870445 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:37.870422 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 12:54:37.871625 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:37.871607 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 12:54:52.282846 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:52.282795 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bf95d5449-p8bwc"] Jul 21 12:54:52.283393 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:52.283371 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31d3bd46-6ad3-44f6-8a5a-652659ca01b1" containerName="manager" Jul 21 12:54:52.283477 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:52.283395 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="31d3bd46-6ad3-44f6-8a5a-652659ca01b1" containerName="manager" Jul 21 12:54:52.283527 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:52.283490 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="31d3bd46-6ad3-44f6-8a5a-652659ca01b1" containerName="manager" Jul 21 12:54:52.287724 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:52.287700 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bf95d5449-p8bwc" Jul 21 12:54:52.289824 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:52.289799 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 21 12:54:52.289958 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:52.289808 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 21 12:54:52.290224 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:52.290208 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7gb5d\"" Jul 21 12:54:52.294889 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:52.294862 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bf95d5449-p8bwc"] Jul 21 12:54:52.302911 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:52.302885 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cee7b5ed-6c3b-4db5-9746-b1f154b77068-webhook-cert\") pod \"maas-controller-6bf95d5449-p8bwc\" (UID: \"cee7b5ed-6c3b-4db5-9746-b1f154b77068\") " pod="opendatahub/maas-controller-6bf95d5449-p8bwc" Jul 21 12:54:52.303058 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:52.302925 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv5zm\" (UniqueName: \"kubernetes.io/projected/cee7b5ed-6c3b-4db5-9746-b1f154b77068-kube-api-access-vv5zm\") pod \"maas-controller-6bf95d5449-p8bwc\" (UID: \"cee7b5ed-6c3b-4db5-9746-b1f154b77068\") " pod="opendatahub/maas-controller-6bf95d5449-p8bwc" Jul 21 12:54:52.403450 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:52.403414 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cee7b5ed-6c3b-4db5-9746-b1f154b77068-webhook-cert\") pod \"maas-controller-6bf95d5449-p8bwc\" (UID: \"cee7b5ed-6c3b-4db5-9746-b1f154b77068\") " pod="opendatahub/maas-controller-6bf95d5449-p8bwc" Jul 21 12:54:52.403450 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:52.403455 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv5zm\" (UniqueName: \"kubernetes.io/projected/cee7b5ed-6c3b-4db5-9746-b1f154b77068-kube-api-access-vv5zm\") pod \"maas-controller-6bf95d5449-p8bwc\" (UID: \"cee7b5ed-6c3b-4db5-9746-b1f154b77068\") " pod="opendatahub/maas-controller-6bf95d5449-p8bwc" Jul 21 12:54:52.406022 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:52.405997 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cee7b5ed-6c3b-4db5-9746-b1f154b77068-webhook-cert\") pod \"maas-controller-6bf95d5449-p8bwc\" (UID: \"cee7b5ed-6c3b-4db5-9746-b1f154b77068\") " pod="opendatahub/maas-controller-6bf95d5449-p8bwc" Jul 21 12:54:52.411987 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:52.411963 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv5zm\" (UniqueName: \"kubernetes.io/projected/cee7b5ed-6c3b-4db5-9746-b1f154b77068-kube-api-access-vv5zm\") pod \"maas-controller-6bf95d5449-p8bwc\" (UID: \"cee7b5ed-6c3b-4db5-9746-b1f154b77068\") " pod="opendatahub/maas-controller-6bf95d5449-p8bwc" Jul 21 12:54:52.600787 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:52.600753 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bf95d5449-p8bwc" Jul 21 12:54:52.953424 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:52.953319 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bf95d5449-p8bwc"] Jul 21 12:54:52.956339 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:54:52.956311 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcee7b5ed_6c3b_4db5_9746_b1f154b77068.slice/crio-0322e7dee72494263994f3e8c9e4c1ca1ce43975e46af7685c2983865a768afe WatchSource:0}: Error finding container 0322e7dee72494263994f3e8c9e4c1ca1ce43975e46af7685c2983865a768afe: Status 404 returned error can't find the container with id 0322e7dee72494263994f3e8c9e4c1ca1ce43975e46af7685c2983865a768afe Jul 21 12:54:53.432950 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:53.432914 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bf95d5449-p8bwc" event={"ID":"cee7b5ed-6c3b-4db5-9746-b1f154b77068","Type":"ContainerStarted","Data":"0322e7dee72494263994f3e8c9e4c1ca1ce43975e46af7685c2983865a768afe"} Jul 21 12:54:54.440046 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:54.440009 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bf95d5449-p8bwc" event={"ID":"cee7b5ed-6c3b-4db5-9746-b1f154b77068","Type":"ContainerStarted","Data":"78bee10491ba385e1534b19dee485092f1680b56037bf0062ebbfdb8f1c4b218"} Jul 21 12:54:54.440507 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:54:54.440169 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bf95d5449-p8bwc" Jul 21 12:55:05.447673 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:05.447639 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bf95d5449-p8bwc" Jul 21 12:55:05.471629 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:05.471556 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bf95d5449-p8bwc" podStartSLOduration=12.828849803 podStartE2EDuration="13.471538805s" totalImagesPullingTime="642.689002ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:54:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:54:54.477190137 +0000 UTC m=+917.095483087" watchObservedRunningTime="2026-07-21 12:55:05.471538805 +0000 UTC m=+928.089831753" Jul 21 12:55:10.114458 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:10.114417 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bf95d5449-p8bwc"] Jul 21 12:55:10.115054 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:10.114629 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6bf95d5449-p8bwc" podUID="cee7b5ed-6c3b-4db5-9746-b1f154b77068" containerName="manager" containerID="cri-o://78bee10491ba385e1534b19dee485092f1680b56037bf0062ebbfdb8f1c4b218" gracePeriod=30 Jul 21 12:55:10.375563 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:10.375496 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bf95d5449-p8bwc" Jul 21 12:55:10.459135 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:10.459096 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cee7b5ed-6c3b-4db5-9746-b1f154b77068-webhook-cert\") pod \"cee7b5ed-6c3b-4db5-9746-b1f154b77068\" (UID: \"cee7b5ed-6c3b-4db5-9746-b1f154b77068\") " Jul 21 12:55:10.459333 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:10.459152 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vv5zm\" (UniqueName: \"kubernetes.io/projected/cee7b5ed-6c3b-4db5-9746-b1f154b77068-kube-api-access-vv5zm\") pod \"cee7b5ed-6c3b-4db5-9746-b1f154b77068\" (UID: \"cee7b5ed-6c3b-4db5-9746-b1f154b77068\") " Jul 21 12:55:10.461436 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:10.461397 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cee7b5ed-6c3b-4db5-9746-b1f154b77068-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "cee7b5ed-6c3b-4db5-9746-b1f154b77068" (UID: "cee7b5ed-6c3b-4db5-9746-b1f154b77068"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 12:55:10.461560 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:10.461473 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cee7b5ed-6c3b-4db5-9746-b1f154b77068-kube-api-access-vv5zm" (OuterVolumeSpecName: "kube-api-access-vv5zm") pod "cee7b5ed-6c3b-4db5-9746-b1f154b77068" (UID: "cee7b5ed-6c3b-4db5-9746-b1f154b77068"). InnerVolumeSpecName "kube-api-access-vv5zm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 12:55:10.537331 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:10.537298 2555 generic.go:356] "Generic (PLEG): container finished" podID="cee7b5ed-6c3b-4db5-9746-b1f154b77068" containerID="78bee10491ba385e1534b19dee485092f1680b56037bf0062ebbfdb8f1c4b218" exitCode=0 Jul 21 12:55:10.537486 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:10.537344 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bf95d5449-p8bwc" Jul 21 12:55:10.537486 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:10.537348 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bf95d5449-p8bwc" event={"ID":"cee7b5ed-6c3b-4db5-9746-b1f154b77068","Type":"ContainerDied","Data":"78bee10491ba385e1534b19dee485092f1680b56037bf0062ebbfdb8f1c4b218"} Jul 21 12:55:10.537486 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:10.537389 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bf95d5449-p8bwc" event={"ID":"cee7b5ed-6c3b-4db5-9746-b1f154b77068","Type":"ContainerDied","Data":"0322e7dee72494263994f3e8c9e4c1ca1ce43975e46af7685c2983865a768afe"} Jul 21 12:55:10.537486 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:10.537410 2555 scope.go:117] "RemoveContainer" containerID="78bee10491ba385e1534b19dee485092f1680b56037bf0062ebbfdb8f1c4b218" Jul 21 12:55:10.547852 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:10.547822 2555 scope.go:117] "RemoveContainer" containerID="78bee10491ba385e1534b19dee485092f1680b56037bf0062ebbfdb8f1c4b218" Jul 21 12:55:10.548171 ip-10-0-133-39 kubenswrapper[2555]: E0721 12:55:10.548148 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78bee10491ba385e1534b19dee485092f1680b56037bf0062ebbfdb8f1c4b218\": container with ID starting with 78bee10491ba385e1534b19dee485092f1680b56037bf0062ebbfdb8f1c4b218 not found: ID does not exist" containerID="78bee10491ba385e1534b19dee485092f1680b56037bf0062ebbfdb8f1c4b218" Jul 21 12:55:10.548281 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:10.548178 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78bee10491ba385e1534b19dee485092f1680b56037bf0062ebbfdb8f1c4b218"} err="failed to get container status \"78bee10491ba385e1534b19dee485092f1680b56037bf0062ebbfdb8f1c4b218\": rpc error: code = NotFound desc = could not find container \"78bee10491ba385e1534b19dee485092f1680b56037bf0062ebbfdb8f1c4b218\": container with ID starting with 78bee10491ba385e1534b19dee485092f1680b56037bf0062ebbfdb8f1c4b218 not found: ID does not exist" Jul 21 12:55:10.559758 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:10.559731 2555 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cee7b5ed-6c3b-4db5-9746-b1f154b77068-webhook-cert\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:55:10.559758 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:10.559757 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vv5zm\" (UniqueName: \"kubernetes.io/projected/cee7b5ed-6c3b-4db5-9746-b1f154b77068-kube-api-access-vv5zm\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 12:55:10.563211 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:10.563175 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bf95d5449-p8bwc"] Jul 21 12:55:10.570487 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:10.570459 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6bf95d5449-p8bwc"] Jul 21 12:55:11.392920 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:11.392882 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bf95d5449-79kjl"] Jul 21 12:55:11.393303 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:11.393219 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cee7b5ed-6c3b-4db5-9746-b1f154b77068" containerName="manager" Jul 21 12:55:11.393303 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:11.393230 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="cee7b5ed-6c3b-4db5-9746-b1f154b77068" containerName="manager" Jul 21 12:55:11.393303 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:11.393298 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="cee7b5ed-6c3b-4db5-9746-b1f154b77068" containerName="manager" Jul 21 12:55:11.397500 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:11.397482 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bf95d5449-79kjl" Jul 21 12:55:11.399731 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:11.399709 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 21 12:55:11.399858 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:11.399713 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7gb5d\"" Jul 21 12:55:11.400143 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:11.400129 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 21 12:55:11.409661 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:11.409637 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bf95d5449-79kjl"] Jul 21 12:55:11.468185 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:11.468138 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtfdq\" (UniqueName: \"kubernetes.io/projected/bdcd5f09-697c-43de-bdc5-da5cebcbcae5-kube-api-access-dtfdq\") pod \"maas-controller-6bf95d5449-79kjl\" (UID: \"bdcd5f09-697c-43de-bdc5-da5cebcbcae5\") " pod="opendatahub/maas-controller-6bf95d5449-79kjl" Jul 21 12:55:11.468387 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:11.468269 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bdcd5f09-697c-43de-bdc5-da5cebcbcae5-webhook-cert\") pod \"maas-controller-6bf95d5449-79kjl\" (UID: \"bdcd5f09-697c-43de-bdc5-da5cebcbcae5\") " pod="opendatahub/maas-controller-6bf95d5449-79kjl" Jul 21 12:55:11.569138 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:11.569092 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bdcd5f09-697c-43de-bdc5-da5cebcbcae5-webhook-cert\") pod \"maas-controller-6bf95d5449-79kjl\" (UID: \"bdcd5f09-697c-43de-bdc5-da5cebcbcae5\") " pod="opendatahub/maas-controller-6bf95d5449-79kjl" Jul 21 12:55:11.569346 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:11.569194 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtfdq\" (UniqueName: \"kubernetes.io/projected/bdcd5f09-697c-43de-bdc5-da5cebcbcae5-kube-api-access-dtfdq\") pod \"maas-controller-6bf95d5449-79kjl\" (UID: \"bdcd5f09-697c-43de-bdc5-da5cebcbcae5\") " pod="opendatahub/maas-controller-6bf95d5449-79kjl" Jul 21 12:55:11.571754 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:11.571726 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bdcd5f09-697c-43de-bdc5-da5cebcbcae5-webhook-cert\") pod \"maas-controller-6bf95d5449-79kjl\" (UID: \"bdcd5f09-697c-43de-bdc5-da5cebcbcae5\") " pod="opendatahub/maas-controller-6bf95d5449-79kjl" Jul 21 12:55:11.579764 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:11.579742 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtfdq\" (UniqueName: \"kubernetes.io/projected/bdcd5f09-697c-43de-bdc5-da5cebcbcae5-kube-api-access-dtfdq\") pod \"maas-controller-6bf95d5449-79kjl\" (UID: \"bdcd5f09-697c-43de-bdc5-da5cebcbcae5\") " pod="opendatahub/maas-controller-6bf95d5449-79kjl" Jul 21 12:55:11.707770 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:11.707681 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bf95d5449-79kjl" Jul 21 12:55:11.858220 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:11.858189 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bf95d5449-79kjl"] Jul 21 12:55:11.860770 ip-10-0-133-39 kubenswrapper[2555]: W0721 12:55:11.860744 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdcd5f09_697c_43de_bdc5_da5cebcbcae5.slice/crio-bd1b3fe624c3243e04c5e46f4a9bdad29383a1b9b948fdd7ceeb26417ec49c83 WatchSource:0}: Error finding container bd1b3fe624c3243e04c5e46f4a9bdad29383a1b9b948fdd7ceeb26417ec49c83: Status 404 returned error can't find the container with id bd1b3fe624c3243e04c5e46f4a9bdad29383a1b9b948fdd7ceeb26417ec49c83 Jul 21 12:55:11.926647 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:11.926614 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cee7b5ed-6c3b-4db5-9746-b1f154b77068" path="/var/lib/kubelet/pods/cee7b5ed-6c3b-4db5-9746-b1f154b77068/volumes" Jul 21 12:55:12.551512 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:12.551459 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bf95d5449-79kjl" event={"ID":"bdcd5f09-697c-43de-bdc5-da5cebcbcae5","Type":"ContainerStarted","Data":"2f92f4b3b27c2b86c4fd924c6823356fd0ebead7047bf0383d2f105195cc090c"} Jul 21 12:55:12.552017 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:12.551533 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bf95d5449-79kjl" Jul 21 12:55:12.552017 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:12.551556 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bf95d5449-79kjl" event={"ID":"bdcd5f09-697c-43de-bdc5-da5cebcbcae5","Type":"ContainerStarted","Data":"bd1b3fe624c3243e04c5e46f4a9bdad29383a1b9b948fdd7ceeb26417ec49c83"} Jul 21 12:55:12.579116 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:12.578908 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bf95d5449-79kjl" podStartSLOduration=1.019495703 podStartE2EDuration="1.578885849s" totalImagesPullingTime="559.390146ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:55:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:55:12.577836683 +0000 UTC m=+935.196129632" watchObservedRunningTime="2026-07-21 12:55:12.578885849 +0000 UTC m=+935.197178795" Jul 21 12:55:23.559292 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:55:23.559246 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bf95d5449-79kjl" Jul 21 12:59:37.898823 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:59:37.898795 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 12:59:37.899704 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:59:37.899667 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 12:59:37.903917 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:59:37.903898 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 12:59:37.904996 ip-10-0-133-39 kubenswrapper[2555]: I0721 12:59:37.904978 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 13:00:07.554709 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:00:07.554671 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq"] Jul 21 13:00:07.555152 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:00:07.554929 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq" podUID="6e614cd5-6b44-4ea6-82fa-e45a6d59394b" containerName="manager" containerID="cri-o://f91b53a6e53cc5cd015cb57c4080efb013f28555e32d795dfdf230341cc41f52" gracePeriod=10 Jul 21 13:00:11.412239 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:00:11.412212 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq" Jul 21 13:00:11.424246 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:00:11.424218 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvw5k\" (UniqueName: \"kubernetes.io/projected/6e614cd5-6b44-4ea6-82fa-e45a6d59394b-kube-api-access-kvw5k\") pod \"6e614cd5-6b44-4ea6-82fa-e45a6d59394b\" (UID: \"6e614cd5-6b44-4ea6-82fa-e45a6d59394b\") " Jul 21 13:00:11.424402 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:00:11.424296 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e614cd5-6b44-4ea6-82fa-e45a6d59394b-extensions-socket-volume\") pod \"6e614cd5-6b44-4ea6-82fa-e45a6d59394b\" (UID: \"6e614cd5-6b44-4ea6-82fa-e45a6d59394b\") " Jul 21 13:00:11.424674 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:00:11.424652 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e614cd5-6b44-4ea6-82fa-e45a6d59394b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6e614cd5-6b44-4ea6-82fa-e45a6d59394b" (UID: "6e614cd5-6b44-4ea6-82fa-e45a6d59394b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:00:11.426447 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:00:11.426423 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e614cd5-6b44-4ea6-82fa-e45a6d59394b-kube-api-access-kvw5k" (OuterVolumeSpecName: "kube-api-access-kvw5k") pod "6e614cd5-6b44-4ea6-82fa-e45a6d59394b" (UID: "6e614cd5-6b44-4ea6-82fa-e45a6d59394b"). InnerVolumeSpecName "kube-api-access-kvw5k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:00:11.524885 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:00:11.524792 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvw5k\" (UniqueName: \"kubernetes.io/projected/6e614cd5-6b44-4ea6-82fa-e45a6d59394b-kube-api-access-kvw5k\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:00:11.524885 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:00:11.524826 2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e614cd5-6b44-4ea6-82fa-e45a6d59394b-extensions-socket-volume\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:00:12.135684 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:00:12.135653 2555 generic.go:356] "Generic (PLEG): container finished" podID="6e614cd5-6b44-4ea6-82fa-e45a6d59394b" containerID="f91b53a6e53cc5cd015cb57c4080efb013f28555e32d795dfdf230341cc41f52" exitCode=0 Jul 21 13:00:12.135829 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:00:12.135703 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq" Jul 21 13:00:12.135829 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:00:12.135737 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq" event={"ID":"6e614cd5-6b44-4ea6-82fa-e45a6d59394b","Type":"ContainerDied","Data":"f91b53a6e53cc5cd015cb57c4080efb013f28555e32d795dfdf230341cc41f52"} Jul 21 13:00:12.135829 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:00:12.135790 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq" event={"ID":"6e614cd5-6b44-4ea6-82fa-e45a6d59394b","Type":"ContainerDied","Data":"cb977491ecee38c74c0649c8dffb1c0535881c66fd1878e6a7b91078b69bd1ba"} Jul 21 13:00:12.135829 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:00:12.135812 2555 scope.go:117] "RemoveContainer" containerID="f91b53a6e53cc5cd015cb57c4080efb013f28555e32d795dfdf230341cc41f52" Jul 21 13:00:12.145893 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:00:12.145872 2555 scope.go:117] "RemoveContainer" containerID="f91b53a6e53cc5cd015cb57c4080efb013f28555e32d795dfdf230341cc41f52" Jul 21 13:00:12.146161 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:00:12.146143 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f91b53a6e53cc5cd015cb57c4080efb013f28555e32d795dfdf230341cc41f52\": container with ID starting with f91b53a6e53cc5cd015cb57c4080efb013f28555e32d795dfdf230341cc41f52 not found: ID does not exist" containerID="f91b53a6e53cc5cd015cb57c4080efb013f28555e32d795dfdf230341cc41f52" Jul 21 13:00:12.146201 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:00:12.146171 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f91b53a6e53cc5cd015cb57c4080efb013f28555e32d795dfdf230341cc41f52"} err="failed to get container status \"f91b53a6e53cc5cd015cb57c4080efb013f28555e32d795dfdf230341cc41f52\": rpc error: code = NotFound desc = could not find container \"f91b53a6e53cc5cd015cb57c4080efb013f28555e32d795dfdf230341cc41f52\": container with ID starting with f91b53a6e53cc5cd015cb57c4080efb013f28555e32d795dfdf230341cc41f52 not found: ID does not exist" Jul 21 13:00:12.156672 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:00:12.156648 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq"] Jul 21 13:00:12.164530 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:00:12.164504 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fqvkq"] Jul 21 13:00:13.925800 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:00:13.925765 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e614cd5-6b44-4ea6-82fa-e45a6d59394b" path="/var/lib/kubelet/pods/6e614cd5-6b44-4ea6-82fa-e45a6d59394b/volumes" Jul 21 13:01:08.691481 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:08.691449 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pf6gh"] Jul 21 13:01:08.691981 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:08.691819 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e614cd5-6b44-4ea6-82fa-e45a6d59394b" containerName="manager" Jul 21 13:01:08.691981 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:08.691830 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e614cd5-6b44-4ea6-82fa-e45a6d59394b" containerName="manager" Jul 21 13:01:08.691981 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:08.691889 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e614cd5-6b44-4ea6-82fa-e45a6d59394b" containerName="manager" Jul 21 13:01:08.695002 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:08.694984 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pf6gh" Jul 21 13:01:08.697520 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:08.697501 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-622jg\"" Jul 21 13:01:08.707787 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:08.707762 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pf6gh"] Jul 21 13:01:08.723993 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:08.723970 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vth7\" (UniqueName: \"kubernetes.io/projected/afd6f2df-034b-45df-811f-0d6c7ec23ac3-kube-api-access-7vth7\") pod \"kuadrant-operator-controller-manager-86878c769f-pf6gh\" (UID: \"afd6f2df-034b-45df-811f-0d6c7ec23ac3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pf6gh" Jul 21 13:01:08.724141 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:08.724016 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/afd6f2df-034b-45df-811f-0d6c7ec23ac3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pf6gh\" (UID: \"afd6f2df-034b-45df-811f-0d6c7ec23ac3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pf6gh" Jul 21 13:01:08.825176 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:08.825138 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vth7\" (UniqueName: \"kubernetes.io/projected/afd6f2df-034b-45df-811f-0d6c7ec23ac3-kube-api-access-7vth7\") pod \"kuadrant-operator-controller-manager-86878c769f-pf6gh\" (UID: \"afd6f2df-034b-45df-811f-0d6c7ec23ac3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pf6gh" Jul 21 13:01:08.825369 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:08.825200 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/afd6f2df-034b-45df-811f-0d6c7ec23ac3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pf6gh\" (UID: \"afd6f2df-034b-45df-811f-0d6c7ec23ac3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pf6gh" Jul 21 13:01:08.825589 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:08.825569 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/afd6f2df-034b-45df-811f-0d6c7ec23ac3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pf6gh\" (UID: \"afd6f2df-034b-45df-811f-0d6c7ec23ac3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pf6gh" Jul 21 13:01:08.836212 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:08.836183 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vth7\" (UniqueName: \"kubernetes.io/projected/afd6f2df-034b-45df-811f-0d6c7ec23ac3-kube-api-access-7vth7\") pod \"kuadrant-operator-controller-manager-86878c769f-pf6gh\" (UID: \"afd6f2df-034b-45df-811f-0d6c7ec23ac3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pf6gh" Jul 21 13:01:09.005074 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:09.004988 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pf6gh" Jul 21 13:01:09.153747 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:09.153712 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pf6gh"] Jul 21 13:01:09.157743 ip-10-0-133-39 kubenswrapper[2555]: W0721 13:01:09.157716 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafd6f2df_034b_45df_811f_0d6c7ec23ac3.slice/crio-dc38b06127fae30ba175d7f519a93d3d6b02e3024a2519a6145d76c88cfa33d4 WatchSource:0}: Error finding container dc38b06127fae30ba175d7f519a93d3d6b02e3024a2519a6145d76c88cfa33d4: Status 404 returned error can't find the container with id dc38b06127fae30ba175d7f519a93d3d6b02e3024a2519a6145d76c88cfa33d4 Jul 21 13:01:09.436515 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:09.436485 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pf6gh" event={"ID":"afd6f2df-034b-45df-811f-0d6c7ec23ac3","Type":"ContainerStarted","Data":"6883202f3f3f271e944a4265426286b69dfab1503e4919436b0516032bf76e9c"} Jul 21 13:01:09.436716 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:09.436536 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pf6gh" event={"ID":"afd6f2df-034b-45df-811f-0d6c7ec23ac3","Type":"ContainerStarted","Data":"dc38b06127fae30ba175d7f519a93d3d6b02e3024a2519a6145d76c88cfa33d4"} Jul 21 13:01:09.436716 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:09.436607 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pf6gh" Jul 21 13:01:09.474738 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:09.474677 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pf6gh" podStartSLOduration=1.474656443 podStartE2EDuration="1.474656443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:01:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:01:09.471861622 +0000 UTC m=+1292.090154572" watchObservedRunningTime="2026-07-21 13:01:09.474656443 +0000 UTC m=+1292.092949391" Jul 21 13:01:20.444552 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:01:20.444520 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pf6gh" Jul 21 13:04:37.934647 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:04:37.934619 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 13:04:37.936169 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:04:37.936144 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 13:04:37.940084 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:04:37.940059 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 13:04:37.941155 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:04:37.941134 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 13:08:13.407781 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:13.407740 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8"] Jul 21 13:08:13.410935 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:13.410915 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" Jul 21 13:08:13.414528 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:13.414505 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 13:08:13.414701 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:13.414544 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e773b8a4-serving-cert\"" Jul 21 13:08:13.415631 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:13.415583 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-njpqk\"" Jul 21 13:08:13.415763 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:13.415745 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 13:08:13.423843 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:13.423817 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8"] Jul 21 13:08:13.533693 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:13.533568 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xw7h\" (UniqueName: \"kubernetes.io/projected/6fb632ac-b02c-4391-bb7c-636b47dd2f12-kube-api-access-9xw7h\") pod \"maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8\" (UID: \"6fb632ac-b02c-4391-bb7c-636b47dd2f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" Jul 21 13:08:13.533888 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:13.533736 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb632ac-b02c-4391-bb7c-636b47dd2f12-maas-api-tls\") pod \"maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8\" (UID: \"6fb632ac-b02c-4391-bb7c-636b47dd2f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" Jul 21 13:08:13.533888 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:13.533801 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb632ac-b02c-4391-bb7c-636b47dd2f12-openshift-service-ca\") pod \"maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8\" (UID: \"6fb632ac-b02c-4391-bb7c-636b47dd2f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" Jul 21 13:08:13.634972 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:13.634935 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb632ac-b02c-4391-bb7c-636b47dd2f12-openshift-service-ca\") pod \"maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8\" (UID: \"6fb632ac-b02c-4391-bb7c-636b47dd2f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" Jul 21 13:08:13.635178 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:13.634985 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xw7h\" (UniqueName: \"kubernetes.io/projected/6fb632ac-b02c-4391-bb7c-636b47dd2f12-kube-api-access-9xw7h\") pod \"maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8\" (UID: \"6fb632ac-b02c-4391-bb7c-636b47dd2f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" Jul 21 13:08:13.635178 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:13.635024 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb632ac-b02c-4391-bb7c-636b47dd2f12-maas-api-tls\") pod \"maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8\" (UID: \"6fb632ac-b02c-4391-bb7c-636b47dd2f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" Jul 21 13:08:13.635178 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:08:13.635129 2555 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-serving-cert: secret "maas-api-e2e-ait-e773b8a4-serving-cert" not found Jul 21 13:08:13.635366 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:08:13.635193 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6fb632ac-b02c-4391-bb7c-636b47dd2f12-maas-api-tls podName:6fb632ac-b02c-4391-bb7c-636b47dd2f12 nodeName:}" failed. No retries permitted until 2026-07-21 13:08:14.135175671 +0000 UTC m=+1716.753468597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6fb632ac-b02c-4391-bb7c-636b47dd2f12-maas-api-tls") pod "maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" (UID: "6fb632ac-b02c-4391-bb7c-636b47dd2f12") : secret "maas-api-e2e-ait-e773b8a4-serving-cert" not found Jul 21 13:08:13.635620 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:13.635583 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb632ac-b02c-4391-bb7c-636b47dd2f12-openshift-service-ca\") pod \"maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8\" (UID: \"6fb632ac-b02c-4391-bb7c-636b47dd2f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" Jul 21 13:08:13.646798 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:13.646770 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xw7h\" (UniqueName: \"kubernetes.io/projected/6fb632ac-b02c-4391-bb7c-636b47dd2f12-kube-api-access-9xw7h\") pod \"maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8\" (UID: \"6fb632ac-b02c-4391-bb7c-636b47dd2f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" Jul 21 13:08:14.141695 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:14.141654 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb632ac-b02c-4391-bb7c-636b47dd2f12-maas-api-tls\") pod \"maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8\" (UID: \"6fb632ac-b02c-4391-bb7c-636b47dd2f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" Jul 21 13:08:14.144444 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:14.144415 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb632ac-b02c-4391-bb7c-636b47dd2f12-maas-api-tls\") pod \"maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8\" (UID: \"6fb632ac-b02c-4391-bb7c-636b47dd2f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" Jul 21 13:08:14.322197 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:14.322163 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" Jul 21 13:08:14.473373 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:14.473193 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8"] Jul 21 13:08:14.476985 ip-10-0-133-39 kubenswrapper[2555]: W0721 13:08:14.476947 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb632ac_b02c_4391_bb7c_636b47dd2f12.slice/crio-ebfd098bc8cd9d830d92924aee26981937febd751d1c8f6c2935de70e03b9097 WatchSource:0}: Error finding container ebfd098bc8cd9d830d92924aee26981937febd751d1c8f6c2935de70e03b9097: Status 404 returned error can't find the container with id ebfd098bc8cd9d830d92924aee26981937febd751d1c8f6c2935de70e03b9097 Jul 21 13:08:14.478449 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:14.478420 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 13:08:14.661247 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:14.661211 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" event={"ID":"6fb632ac-b02c-4391-bb7c-636b47dd2f12","Type":"ContainerStarted","Data":"ebfd098bc8cd9d830d92924aee26981937febd751d1c8f6c2935de70e03b9097"} Jul 21 13:08:16.674274 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:16.674236 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" event={"ID":"6fb632ac-b02c-4391-bb7c-636b47dd2f12","Type":"ContainerStarted","Data":"a7f4ed2255a8559ff0a37cb27361093a4df86b8a9ace3339d83a731a4089fb89"} Jul 21 13:08:16.674714 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:16.674354 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" Jul 21 13:08:22.686097 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:22.686067 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" Jul 21 13:08:22.710474 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:22.710402 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" podStartSLOduration=8.144501504 podStartE2EDuration="9.710385527s" totalImagesPullingTime="1.565884023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:08:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:08:16.69690611 +0000 UTC m=+1719.315199059" watchObservedRunningTime="2026-07-21 13:08:22.710385527 +0000 UTC m=+1725.328678472" Jul 21 13:08:49.408129 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.408094 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8"] Jul 21 13:08:49.408866 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.408358 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" podUID="6fb632ac-b02c-4391-bb7c-636b47dd2f12" containerName="maas-api" containerID="cri-o://a7f4ed2255a8559ff0a37cb27361093a4df86b8a9ace3339d83a731a4089fb89" gracePeriod=30 Jul 21 13:08:49.669291 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.669223 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" Jul 21 13:08:49.762047 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.762012 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xw7h\" (UniqueName: \"kubernetes.io/projected/6fb632ac-b02c-4391-bb7c-636b47dd2f12-kube-api-access-9xw7h\") pod \"6fb632ac-b02c-4391-bb7c-636b47dd2f12\" (UID: \"6fb632ac-b02c-4391-bb7c-636b47dd2f12\") " Jul 21 13:08:49.762245 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.762078 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb632ac-b02c-4391-bb7c-636b47dd2f12-openshift-service-ca\") pod \"6fb632ac-b02c-4391-bb7c-636b47dd2f12\" (UID: \"6fb632ac-b02c-4391-bb7c-636b47dd2f12\") " Jul 21 13:08:49.762245 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.762118 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb632ac-b02c-4391-bb7c-636b47dd2f12-maas-api-tls\") pod \"6fb632ac-b02c-4391-bb7c-636b47dd2f12\" (UID: \"6fb632ac-b02c-4391-bb7c-636b47dd2f12\") " Jul 21 13:08:49.762398 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.762371 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fb632ac-b02c-4391-bb7c-636b47dd2f12-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6fb632ac-b02c-4391-bb7c-636b47dd2f12" (UID: "6fb632ac-b02c-4391-bb7c-636b47dd2f12"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:08:49.764482 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.764451 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fb632ac-b02c-4391-bb7c-636b47dd2f12-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6fb632ac-b02c-4391-bb7c-636b47dd2f12" (UID: "6fb632ac-b02c-4391-bb7c-636b47dd2f12"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:08:49.764637 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.764535 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fb632ac-b02c-4391-bb7c-636b47dd2f12-kube-api-access-9xw7h" (OuterVolumeSpecName: "kube-api-access-9xw7h") pod "6fb632ac-b02c-4391-bb7c-636b47dd2f12" (UID: "6fb632ac-b02c-4391-bb7c-636b47dd2f12"). InnerVolumeSpecName "kube-api-access-9xw7h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:08:49.859897 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.859866 2555 generic.go:356] "Generic (PLEG): container finished" podID="6fb632ac-b02c-4391-bb7c-636b47dd2f12" containerID="a7f4ed2255a8559ff0a37cb27361093a4df86b8a9ace3339d83a731a4089fb89" exitCode=0 Jul 21 13:08:49.860079 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.859922 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" Jul 21 13:08:49.860079 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.859957 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" event={"ID":"6fb632ac-b02c-4391-bb7c-636b47dd2f12","Type":"ContainerDied","Data":"a7f4ed2255a8559ff0a37cb27361093a4df86b8a9ace3339d83a731a4089fb89"} Jul 21 13:08:49.860079 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.860038 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8" event={"ID":"6fb632ac-b02c-4391-bb7c-636b47dd2f12","Type":"ContainerDied","Data":"ebfd098bc8cd9d830d92924aee26981937febd751d1c8f6c2935de70e03b9097"} Jul 21 13:08:49.860079 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.860067 2555 scope.go:117] "RemoveContainer" containerID="a7f4ed2255a8559ff0a37cb27361093a4df86b8a9ace3339d83a731a4089fb89" Jul 21 13:08:49.862746 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.862725 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb632ac-b02c-4391-bb7c-636b47dd2f12-maas-api-tls\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:08:49.862876 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.862763 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xw7h\" (UniqueName: \"kubernetes.io/projected/6fb632ac-b02c-4391-bb7c-636b47dd2f12-kube-api-access-9xw7h\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:08:49.862876 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.862773 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb632ac-b02c-4391-bb7c-636b47dd2f12-openshift-service-ca\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:08:49.869438 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.869417 2555 scope.go:117] "RemoveContainer" containerID="a7f4ed2255a8559ff0a37cb27361093a4df86b8a9ace3339d83a731a4089fb89" Jul 21 13:08:49.869777 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:08:49.869760 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7f4ed2255a8559ff0a37cb27361093a4df86b8a9ace3339d83a731a4089fb89\": container with ID starting with a7f4ed2255a8559ff0a37cb27361093a4df86b8a9ace3339d83a731a4089fb89 not found: ID does not exist" containerID="a7f4ed2255a8559ff0a37cb27361093a4df86b8a9ace3339d83a731a4089fb89" Jul 21 13:08:49.869822 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.869787 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7f4ed2255a8559ff0a37cb27361093a4df86b8a9ace3339d83a731a4089fb89"} err="failed to get container status \"a7f4ed2255a8559ff0a37cb27361093a4df86b8a9ace3339d83a731a4089fb89\": rpc error: code = NotFound desc = could not find container \"a7f4ed2255a8559ff0a37cb27361093a4df86b8a9ace3339d83a731a4089fb89\": container with ID starting with a7f4ed2255a8559ff0a37cb27361093a4df86b8a9ace3339d83a731a4089fb89 not found: ID does not exist" Jul 21 13:08:49.883958 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.883925 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8"] Jul 21 13:08:49.888397 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.888370 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e773b8a4-f466ffd95-ks7p8"] Jul 21 13:08:49.926408 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:49.926325 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fb632ac-b02c-4391-bb7c-636b47dd2f12" path="/var/lib/kubelet/pods/6fb632ac-b02c-4391-bb7c-636b47dd2f12/volumes" Jul 21 13:08:55.107925 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.107888 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr"] Jul 21 13:08:55.108468 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.108450 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fb632ac-b02c-4391-bb7c-636b47dd2f12" containerName="maas-api" Jul 21 13:08:55.108537 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.108469 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fb632ac-b02c-4391-bb7c-636b47dd2f12" containerName="maas-api" Jul 21 13:08:55.108610 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.108550 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fb632ac-b02c-4391-bb7c-636b47dd2f12" containerName="maas-api" Jul 21 13:08:55.112825 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.112802 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.115426 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.115408 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-f454aa60-gw-openshift-default-dockercfg-ft789\"" Jul 21 13:08:55.123699 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.123674 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr"] Jul 21 13:08:55.213976 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.213936 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-podinfo\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.213976 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.213977 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-credential-socket\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.214173 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.214017 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-envoy\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.214173 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.214071 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-workload-certs\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.214173 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.214099 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-token\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.214173 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.214137 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce3e45b2-3d0f-4799-94cc-510424eb1874-istiod-ca-cert\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.214305 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.214180 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5smq6\" (UniqueName: \"kubernetes.io/projected/ce3e45b2-3d0f-4799-94cc-510424eb1874-kube-api-access-5smq6\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.214305 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.214239 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-workload-socket\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.214305 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.214261 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-data\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.314861 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.314823 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce3e45b2-3d0f-4799-94cc-510424eb1874-istiod-ca-cert\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.314861 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.314864 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5smq6\" (UniqueName: \"kubernetes.io/projected/ce3e45b2-3d0f-4799-94cc-510424eb1874-kube-api-access-5smq6\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.315132 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.314895 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-workload-socket\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.315132 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.314910 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-data\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.315132 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.314936 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-podinfo\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.315132 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.314970 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-credential-socket\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.315132 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.315003 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-envoy\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.315132 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.315035 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-workload-certs\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.315132 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.315060 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-token\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.315498 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.315352 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-workload-socket\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.315498 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.315366 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-credential-socket\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.315643 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.315529 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-data\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.315694 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.315659 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce3e45b2-3d0f-4799-94cc-510424eb1874-istiod-ca-cert\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.315728 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.315691 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-workload-certs\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.317505 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.317482 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-envoy\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.317875 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.317853 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-podinfo\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.324719 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.324689 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-token\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.324822 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.324746 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5smq6\" (UniqueName: \"kubernetes.io/projected/ce3e45b2-3d0f-4799-94cc-510424eb1874-kube-api-access-5smq6\") pod \"e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.426528 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.426444 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:55.585369 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.585342 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr"] Jul 21 13:08:55.588965 ip-10-0-133-39 kubenswrapper[2555]: W0721 13:08:55.588927 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce3e45b2_3d0f_4799_94cc_510424eb1874.slice/crio-05cdbc4ecd8dcde58500d0382605b530185e5338733a89b4772bc5bd4c76baeb WatchSource:0}: Error finding container 05cdbc4ecd8dcde58500d0382605b530185e5338733a89b4772bc5bd4c76baeb: Status 404 returned error can't find the container with id 05cdbc4ecd8dcde58500d0382605b530185e5338733a89b4772bc5bd4c76baeb Jul 21 13:08:55.894381 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:55.894340 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" event={"ID":"ce3e45b2-3d0f-4799-94cc-510424eb1874","Type":"ContainerStarted","Data":"05cdbc4ecd8dcde58500d0382605b530185e5338733a89b4772bc5bd4c76baeb"} Jul 21 13:08:58.427758 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:58.427718 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:08:58.428160 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:58.427814 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:08:58.428160 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:58.427859 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:08:58.912889 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:58.912856 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" event={"ID":"ce3e45b2-3d0f-4799-94cc-510424eb1874","Type":"ContainerStarted","Data":"fd6032a95fed00d8acdd534dbf017a3549134ba2380acebc1165c36f19dd46c2"} Jul 21 13:08:58.939100 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:58.939039 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" podStartSLOduration=1.103321393 podStartE2EDuration="3.939022155s" totalImagesPullingTime="2.835700762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:08:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:08:58.936656678 +0000 UTC m=+1761.554949627" watchObservedRunningTime="2026-07-21 13:08:58.939022155 +0000 UTC m=+1761.557315102" Jul 21 13:08:59.427466 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:59.427428 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:59.432663 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:59.432630 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:59.919426 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:59.919389 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:08:59.925469 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:08:59.925444 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:09:12.990360 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:12.990271 2555 generic.go:356] "Generic (PLEG): container finished" podID="ce3e45b2-3d0f-4799-94cc-510424eb1874" containerID="fd6032a95fed00d8acdd534dbf017a3549134ba2380acebc1165c36f19dd46c2" exitCode=0 Jul 21 13:09:12.990360 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:12.990347 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" event={"ID":"ce3e45b2-3d0f-4799-94cc-510424eb1874","Type":"ContainerDied","Data":"fd6032a95fed00d8acdd534dbf017a3549134ba2380acebc1165c36f19dd46c2"} Jul 21 13:09:12.990885 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:12.990746 2555 scope.go:117] "RemoveContainer" containerID="fd6032a95fed00d8acdd534dbf017a3549134ba2380acebc1165c36f19dd46c2" Jul 21 13:09:12.991705 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:12.991677 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:09:12.991762 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:12.991744 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:09:12.991797 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:12.991776 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:09:13.998583 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:13.998549 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" event={"ID":"ce3e45b2-3d0f-4799-94cc-510424eb1874","Type":"ContainerStarted","Data":"2a90bfab6aad42b433fec719e99e8a8ed63af2c59c93d3835a472cd71a005863"} Jul 21 13:09:14.427035 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:14.426991 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:09:14.428449 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:14.428416 2555 patch_prober.go:28] interesting pod/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 21 13:09:14.428741 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:14.428711 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" podUID="ce3e45b2-3d0f-4799-94cc-510424eb1874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 21 13:09:14.919777 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:14.919733 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:09:15.427544 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:15.427506 2555 patch_prober.go:28] interesting pod/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 21 13:09:15.428202 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:15.428167 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" podUID="ce3e45b2-3d0f-4799-94cc-510424eb1874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 21 13:09:16.427505 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:16.427461 2555 patch_prober.go:28] interesting pod/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 21 13:09:16.427749 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:16.427551 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" podUID="ce3e45b2-3d0f-4799-94cc-510424eb1874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 21 13:09:17.427559 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:17.427518 2555 patch_prober.go:28] interesting pod/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 21 13:09:17.427897 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:17.427860 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" podUID="ce3e45b2-3d0f-4799-94cc-510424eb1874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 21 13:09:18.431383 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:18.431350 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:09:18.432643 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:18.432619 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:09:37.969056 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:37.969030 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 13:09:37.971608 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:37.971569 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 13:09:37.974502 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:37.974479 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 13:09:37.977257 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:37.977222 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 13:09:40.690507 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:40.690470 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr"] Jul 21 13:09:40.690967 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:40.690724 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" podUID="ce3e45b2-3d0f-4799-94cc-510424eb1874" containerName="istio-proxy" containerID="cri-o://2a90bfab6aad42b433fec719e99e8a8ed63af2c59c93d3835a472cd71a005863" gracePeriod=30 Jul 21 13:09:43.952105 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:43.952071 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5"] Jul 21 13:09:43.956929 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:43.956902 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" Jul 21 13:09:43.959611 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:43.959566 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 13:09:43.959611 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:43.959610 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 13:09:43.959801 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:43.959618 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-1f53c0d5-serving-cert\"" Jul 21 13:09:43.959801 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:43.959656 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-njpqk\"" Jul 21 13:09:43.968265 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:43.968188 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5"] Jul 21 13:09:44.044867 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:44.044831 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a28e34d2-a1f1-440b-9554-1024e56a3819-maas-api-tls\") pod \"maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5\" (UID: \"a28e34d2-a1f1-440b-9554-1024e56a3819\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" Jul 21 13:09:44.044867 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:44.044871 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a28e34d2-a1f1-440b-9554-1024e56a3819-openshift-service-ca\") pod \"maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5\" (UID: \"a28e34d2-a1f1-440b-9554-1024e56a3819\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" Jul 21 13:09:44.045112 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:44.045083 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfg5h\" (UniqueName: \"kubernetes.io/projected/a28e34d2-a1f1-440b-9554-1024e56a3819-kube-api-access-kfg5h\") pod \"maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5\" (UID: \"a28e34d2-a1f1-440b-9554-1024e56a3819\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" Jul 21 13:09:44.145822 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:44.145790 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfg5h\" (UniqueName: \"kubernetes.io/projected/a28e34d2-a1f1-440b-9554-1024e56a3819-kube-api-access-kfg5h\") pod \"maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5\" (UID: \"a28e34d2-a1f1-440b-9554-1024e56a3819\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" Jul 21 13:09:44.146012 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:44.145849 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a28e34d2-a1f1-440b-9554-1024e56a3819-maas-api-tls\") pod \"maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5\" (UID: \"a28e34d2-a1f1-440b-9554-1024e56a3819\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" Jul 21 13:09:44.146012 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:44.145876 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a28e34d2-a1f1-440b-9554-1024e56a3819-openshift-service-ca\") pod \"maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5\" (UID: \"a28e34d2-a1f1-440b-9554-1024e56a3819\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" Jul 21 13:09:44.146512 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:44.146490 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a28e34d2-a1f1-440b-9554-1024e56a3819-openshift-service-ca\") pod \"maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5\" (UID: \"a28e34d2-a1f1-440b-9554-1024e56a3819\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" Jul 21 13:09:44.148434 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:44.148413 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a28e34d2-a1f1-440b-9554-1024e56a3819-maas-api-tls\") pod \"maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5\" (UID: \"a28e34d2-a1f1-440b-9554-1024e56a3819\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" Jul 21 13:09:44.158504 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:44.158475 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfg5h\" (UniqueName: \"kubernetes.io/projected/a28e34d2-a1f1-440b-9554-1024e56a3819-kube-api-access-kfg5h\") pod \"maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5\" (UID: \"a28e34d2-a1f1-440b-9554-1024e56a3819\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" Jul 21 13:09:44.271322 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:44.271221 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" Jul 21 13:09:44.428525 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:44.428498 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5"] Jul 21 13:09:44.431413 ip-10-0-133-39 kubenswrapper[2555]: W0721 13:09:44.431387 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda28e34d2_a1f1_440b_9554_1024e56a3819.slice/crio-16f019b2f592a9fc6b9a5354d1c9e32e81df64dd25a327118bf36e088d661228 WatchSource:0}: Error finding container 16f019b2f592a9fc6b9a5354d1c9e32e81df64dd25a327118bf36e088d661228: Status 404 returned error can't find the container with id 16f019b2f592a9fc6b9a5354d1c9e32e81df64dd25a327118bf36e088d661228 Jul 21 13:09:45.179091 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:45.179053 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" event={"ID":"a28e34d2-a1f1-440b-9554-1024e56a3819","Type":"ContainerStarted","Data":"5abca516e7bd5c2b040ab811eb78ee4b6e1bff1c5abcf779c5aea8a1c21698b4"} Jul 21 13:09:45.179521 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:45.179162 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" Jul 21 13:09:45.179521 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:45.179191 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" event={"ID":"a28e34d2-a1f1-440b-9554-1024e56a3819","Type":"ContainerStarted","Data":"16f019b2f592a9fc6b9a5354d1c9e32e81df64dd25a327118bf36e088d661228"} Jul 21 13:09:45.204073 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:45.204013 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" podStartSLOduration=1.877849979 podStartE2EDuration="2.203994418s" totalImagesPullingTime="326.144439ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:09:45.201143558 +0000 UTC m=+1807.819436506" watchObservedRunningTime="2026-07-21 13:09:45.203994418 +0000 UTC m=+1807.822287366" Jul 21 13:09:45.953812 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:45.953738 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:09:46.063573 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.063528 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-credential-socket\") pod \"ce3e45b2-3d0f-4799-94cc-510424eb1874\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " Jul 21 13:09:46.063807 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.063583 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-token\") pod \"ce3e45b2-3d0f-4799-94cc-510424eb1874\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " Jul 21 13:09:46.063807 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.063637 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5smq6\" (UniqueName: \"kubernetes.io/projected/ce3e45b2-3d0f-4799-94cc-510424eb1874-kube-api-access-5smq6\") pod \"ce3e45b2-3d0f-4799-94cc-510424eb1874\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " Jul 21 13:09:46.063807 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.063662 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-workload-certs\") pod \"ce3e45b2-3d0f-4799-94cc-510424eb1874\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " Jul 21 13:09:46.063975 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.063846 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-podinfo\") pod \"ce3e45b2-3d0f-4799-94cc-510424eb1874\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " Jul 21 13:09:46.063975 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.063894 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-envoy\") pod \"ce3e45b2-3d0f-4799-94cc-510424eb1874\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " Jul 21 13:09:46.063975 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.063919 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ce3e45b2-3d0f-4799-94cc-510424eb1874" (UID: "ce3e45b2-3d0f-4799-94cc-510424eb1874"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:09:46.063975 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.063923 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce3e45b2-3d0f-4799-94cc-510424eb1874-istiod-ca-cert\") pod \"ce3e45b2-3d0f-4799-94cc-510424eb1874\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " Jul 21 13:09:46.063975 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.063963 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-data\") pod \"ce3e45b2-3d0f-4799-94cc-510424eb1874\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " Jul 21 13:09:46.064236 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.064007 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ce3e45b2-3d0f-4799-94cc-510424eb1874" (UID: "ce3e45b2-3d0f-4799-94cc-510424eb1874"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:09:46.064236 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.064021 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-workload-socket\") pod \"ce3e45b2-3d0f-4799-94cc-510424eb1874\" (UID: \"ce3e45b2-3d0f-4799-94cc-510424eb1874\") " Jul 21 13:09:46.064236 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.064223 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ce3e45b2-3d0f-4799-94cc-510424eb1874" (UID: "ce3e45b2-3d0f-4799-94cc-510424eb1874"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:09:46.064424 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.064405 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-credential-socket\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:09:46.064482 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.064428 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-workload-certs\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:09:46.064482 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.064442 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-workload-socket\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:09:46.064482 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.064456 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce3e45b2-3d0f-4799-94cc-510424eb1874-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ce3e45b2-3d0f-4799-94cc-510424eb1874" (UID: "ce3e45b2-3d0f-4799-94cc-510424eb1874"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:09:46.065214 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.065188 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-data" (OuterVolumeSpecName: "istio-data") pod "ce3e45b2-3d0f-4799-94cc-510424eb1874" (UID: "ce3e45b2-3d0f-4799-94cc-510424eb1874"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:09:46.066417 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.066388 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ce3e45b2-3d0f-4799-94cc-510424eb1874" (UID: "ce3e45b2-3d0f-4799-94cc-510424eb1874"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 13:09:46.066417 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.066403 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-token" (OuterVolumeSpecName: "istio-token") pod "ce3e45b2-3d0f-4799-94cc-510424eb1874" (UID: "ce3e45b2-3d0f-4799-94cc-510424eb1874"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:09:46.066417 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.066403 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce3e45b2-3d0f-4799-94cc-510424eb1874-kube-api-access-5smq6" (OuterVolumeSpecName: "kube-api-access-5smq6") pod "ce3e45b2-3d0f-4799-94cc-510424eb1874" (UID: "ce3e45b2-3d0f-4799-94cc-510424eb1874"). InnerVolumeSpecName "kube-api-access-5smq6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:09:46.066584 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.066415 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ce3e45b2-3d0f-4799-94cc-510424eb1874" (UID: "ce3e45b2-3d0f-4799-94cc-510424eb1874"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:09:46.165109 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.165071 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-token\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:09:46.165109 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.165103 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5smq6\" (UniqueName: \"kubernetes.io/projected/ce3e45b2-3d0f-4799-94cc-510424eb1874-kube-api-access-5smq6\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:09:46.165109 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.165113 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-podinfo\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:09:46.165346 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.165123 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-envoy\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:09:46.165346 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.165132 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce3e45b2-3d0f-4799-94cc-510424eb1874-istiod-ca-cert\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:09:46.165346 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.165139 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce3e45b2-3d0f-4799-94cc-510424eb1874-istio-data\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:09:46.186397 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.186359 2555 generic.go:356] "Generic (PLEG): container finished" podID="ce3e45b2-3d0f-4799-94cc-510424eb1874" containerID="2a90bfab6aad42b433fec719e99e8a8ed63af2c59c93d3835a472cd71a005863" exitCode=0 Jul 21 13:09:46.186889 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.186408 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" Jul 21 13:09:46.186889 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.186438 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" event={"ID":"ce3e45b2-3d0f-4799-94cc-510424eb1874","Type":"ContainerDied","Data":"2a90bfab6aad42b433fec719e99e8a8ed63af2c59c93d3835a472cd71a005863"} Jul 21 13:09:46.186889 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.186489 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr" event={"ID":"ce3e45b2-3d0f-4799-94cc-510424eb1874","Type":"ContainerDied","Data":"05cdbc4ecd8dcde58500d0382605b530185e5338733a89b4772bc5bd4c76baeb"} Jul 21 13:09:46.186889 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.186512 2555 scope.go:117] "RemoveContainer" containerID="2a90bfab6aad42b433fec719e99e8a8ed63af2c59c93d3835a472cd71a005863" Jul 21 13:09:46.196741 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.196717 2555 scope.go:117] "RemoveContainer" containerID="fd6032a95fed00d8acdd534dbf017a3549134ba2380acebc1165c36f19dd46c2" Jul 21 13:09:46.205507 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.205488 2555 scope.go:117] "RemoveContainer" containerID="2a90bfab6aad42b433fec719e99e8a8ed63af2c59c93d3835a472cd71a005863" Jul 21 13:09:46.205800 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:09:46.205780 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a90bfab6aad42b433fec719e99e8a8ed63af2c59c93d3835a472cd71a005863\": container with ID starting with 2a90bfab6aad42b433fec719e99e8a8ed63af2c59c93d3835a472cd71a005863 not found: ID does not exist" containerID="2a90bfab6aad42b433fec719e99e8a8ed63af2c59c93d3835a472cd71a005863" Jul 21 13:09:46.205887 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.205809 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a90bfab6aad42b433fec719e99e8a8ed63af2c59c93d3835a472cd71a005863"} err="failed to get container status \"2a90bfab6aad42b433fec719e99e8a8ed63af2c59c93d3835a472cd71a005863\": rpc error: code = NotFound desc = could not find container \"2a90bfab6aad42b433fec719e99e8a8ed63af2c59c93d3835a472cd71a005863\": container with ID starting with 2a90bfab6aad42b433fec719e99e8a8ed63af2c59c93d3835a472cd71a005863 not found: ID does not exist" Jul 21 13:09:46.205887 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.205827 2555 scope.go:117] "RemoveContainer" containerID="fd6032a95fed00d8acdd534dbf017a3549134ba2380acebc1165c36f19dd46c2" Jul 21 13:09:46.206086 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:09:46.206067 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd6032a95fed00d8acdd534dbf017a3549134ba2380acebc1165c36f19dd46c2\": container with ID starting with fd6032a95fed00d8acdd534dbf017a3549134ba2380acebc1165c36f19dd46c2 not found: ID does not exist" containerID="fd6032a95fed00d8acdd534dbf017a3549134ba2380acebc1165c36f19dd46c2" Jul 21 13:09:46.206128 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.206091 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd6032a95fed00d8acdd534dbf017a3549134ba2380acebc1165c36f19dd46c2"} err="failed to get container status \"fd6032a95fed00d8acdd534dbf017a3549134ba2380acebc1165c36f19dd46c2\": rpc error: code = NotFound desc = could not find container \"fd6032a95fed00d8acdd534dbf017a3549134ba2380acebc1165c36f19dd46c2\": container with ID starting with fd6032a95fed00d8acdd534dbf017a3549134ba2380acebc1165c36f19dd46c2 not found: ID does not exist" Jul 21 13:09:46.209308 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.209269 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr"] Jul 21 13:09:46.216238 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:46.216207 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-f454aa60-gw-openshift-default-86787778c5-bfrhr"] Jul 21 13:09:47.927043 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:47.927009 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce3e45b2-3d0f-4799-94cc-510424eb1874" path="/var/lib/kubelet/pods/ce3e45b2-3d0f-4799-94cc-510424eb1874/volumes" Jul 21 13:09:51.192823 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:09:51.192793 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" Jul 21 13:10:20.380653 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:20.380614 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5"] Jul 21 13:10:20.381112 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:20.380898 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" podUID="a28e34d2-a1f1-440b-9554-1024e56a3819" containerName="maas-api" containerID="cri-o://5abca516e7bd5c2b040ab811eb78ee4b6e1bff1c5abcf779c5aea8a1c21698b4" gracePeriod=30 Jul 21 13:10:20.633649 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:20.633565 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" Jul 21 13:10:20.767083 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:20.767047 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a28e34d2-a1f1-440b-9554-1024e56a3819-openshift-service-ca\") pod \"a28e34d2-a1f1-440b-9554-1024e56a3819\" (UID: \"a28e34d2-a1f1-440b-9554-1024e56a3819\") " Jul 21 13:10:20.767277 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:20.767121 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfg5h\" (UniqueName: \"kubernetes.io/projected/a28e34d2-a1f1-440b-9554-1024e56a3819-kube-api-access-kfg5h\") pod \"a28e34d2-a1f1-440b-9554-1024e56a3819\" (UID: \"a28e34d2-a1f1-440b-9554-1024e56a3819\") " Jul 21 13:10:20.767277 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:20.767243 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a28e34d2-a1f1-440b-9554-1024e56a3819-maas-api-tls\") pod \"a28e34d2-a1f1-440b-9554-1024e56a3819\" (UID: \"a28e34d2-a1f1-440b-9554-1024e56a3819\") " Jul 21 13:10:20.767487 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:20.767463 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a28e34d2-a1f1-440b-9554-1024e56a3819-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a28e34d2-a1f1-440b-9554-1024e56a3819" (UID: "a28e34d2-a1f1-440b-9554-1024e56a3819"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:10:20.769386 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:20.769361 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a28e34d2-a1f1-440b-9554-1024e56a3819-kube-api-access-kfg5h" (OuterVolumeSpecName: "kube-api-access-kfg5h") pod "a28e34d2-a1f1-440b-9554-1024e56a3819" (UID: "a28e34d2-a1f1-440b-9554-1024e56a3819"). InnerVolumeSpecName "kube-api-access-kfg5h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:10:20.769498 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:20.769407 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a28e34d2-a1f1-440b-9554-1024e56a3819-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a28e34d2-a1f1-440b-9554-1024e56a3819" (UID: "a28e34d2-a1f1-440b-9554-1024e56a3819"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:10:20.868819 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:20.868782 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfg5h\" (UniqueName: \"kubernetes.io/projected/a28e34d2-a1f1-440b-9554-1024e56a3819-kube-api-access-kfg5h\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:10:20.868819 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:20.868815 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a28e34d2-a1f1-440b-9554-1024e56a3819-maas-api-tls\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:10:20.868819 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:20.868825 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a28e34d2-a1f1-440b-9554-1024e56a3819-openshift-service-ca\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:10:21.379587 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:21.379550 2555 generic.go:356] "Generic (PLEG): container finished" podID="a28e34d2-a1f1-440b-9554-1024e56a3819" containerID="5abca516e7bd5c2b040ab811eb78ee4b6e1bff1c5abcf779c5aea8a1c21698b4" exitCode=0 Jul 21 13:10:21.379868 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:21.379623 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" Jul 21 13:10:21.379868 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:21.379647 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" event={"ID":"a28e34d2-a1f1-440b-9554-1024e56a3819","Type":"ContainerDied","Data":"5abca516e7bd5c2b040ab811eb78ee4b6e1bff1c5abcf779c5aea8a1c21698b4"} Jul 21 13:10:21.379868 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:21.379715 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5" event={"ID":"a28e34d2-a1f1-440b-9554-1024e56a3819","Type":"ContainerDied","Data":"16f019b2f592a9fc6b9a5354d1c9e32e81df64dd25a327118bf36e088d661228"} Jul 21 13:10:21.379868 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:21.379745 2555 scope.go:117] "RemoveContainer" containerID="5abca516e7bd5c2b040ab811eb78ee4b6e1bff1c5abcf779c5aea8a1c21698b4" Jul 21 13:10:21.389706 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:21.389534 2555 scope.go:117] "RemoveContainer" containerID="5abca516e7bd5c2b040ab811eb78ee4b6e1bff1c5abcf779c5aea8a1c21698b4" Jul 21 13:10:21.389963 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:10:21.389943 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5abca516e7bd5c2b040ab811eb78ee4b6e1bff1c5abcf779c5aea8a1c21698b4\": container with ID starting with 5abca516e7bd5c2b040ab811eb78ee4b6e1bff1c5abcf779c5aea8a1c21698b4 not found: ID does not exist" containerID="5abca516e7bd5c2b040ab811eb78ee4b6e1bff1c5abcf779c5aea8a1c21698b4" Jul 21 13:10:21.390010 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:21.389974 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5abca516e7bd5c2b040ab811eb78ee4b6e1bff1c5abcf779c5aea8a1c21698b4"} err="failed to get container status \"5abca516e7bd5c2b040ab811eb78ee4b6e1bff1c5abcf779c5aea8a1c21698b4\": rpc error: code = NotFound desc = could not find container \"5abca516e7bd5c2b040ab811eb78ee4b6e1bff1c5abcf779c5aea8a1c21698b4\": container with ID starting with 5abca516e7bd5c2b040ab811eb78ee4b6e1bff1c5abcf779c5aea8a1c21698b4 not found: ID does not exist" Jul 21 13:10:21.404469 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:21.404438 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5"] Jul 21 13:10:21.410927 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:21.410900 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1f53c0d5-777d4b6f69-jc9c5"] Jul 21 13:10:21.926926 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:21.926889 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a28e34d2-a1f1-440b-9554-1024e56a3819" path="/var/lib/kubelet/pods/a28e34d2-a1f1-440b-9554-1024e56a3819/volumes" Jul 21 13:10:26.708894 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.708852 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww"] Jul 21 13:10:26.709371 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.709229 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a28e34d2-a1f1-440b-9554-1024e56a3819" containerName="maas-api" Jul 21 13:10:26.709371 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.709241 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a28e34d2-a1f1-440b-9554-1024e56a3819" containerName="maas-api" Jul 21 13:10:26.709371 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.709258 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce3e45b2-3d0f-4799-94cc-510424eb1874" containerName="istio-proxy" Jul 21 13:10:26.709371 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.709263 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce3e45b2-3d0f-4799-94cc-510424eb1874" containerName="istio-proxy" Jul 21 13:10:26.709371 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.709275 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce3e45b2-3d0f-4799-94cc-510424eb1874" containerName="istio-proxy" Jul 21 13:10:26.709371 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.709281 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce3e45b2-3d0f-4799-94cc-510424eb1874" containerName="istio-proxy" Jul 21 13:10:26.709371 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.709338 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a28e34d2-a1f1-440b-9554-1024e56a3819" containerName="maas-api" Jul 21 13:10:26.709371 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.709345 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce3e45b2-3d0f-4799-94cc-510424eb1874" containerName="istio-proxy" Jul 21 13:10:26.709371 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.709352 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce3e45b2-3d0f-4799-94cc-510424eb1874" containerName="istio-proxy" Jul 21 13:10:26.713842 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.713821 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.716112 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.716088 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-c06656ac-openshift-default-dockercfg-6p6qf\"" Jul 21 13:10:26.725274 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.725242 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww"] Jul 21 13:10:26.823437 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.823400 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-podinfo\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.823656 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.823457 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-workload-socket\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.823656 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.823484 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2jxq\" (UniqueName: \"kubernetes.io/projected/23cd5145-8ead-40fd-9021-f00dc677eb3b-kube-api-access-h2jxq\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.823656 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.823539 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-workload-certs\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.823656 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.823577 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-envoy\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.823808 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.823682 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-credential-socket\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.823808 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.823729 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23cd5145-8ead-40fd-9021-f00dc677eb3b-istiod-ca-cert\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.823808 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.823782 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-data\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.823808 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.823801 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-token\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.924882 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.924846 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-workload-certs\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.925109 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.924897 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-envoy\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.925109 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.924919 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-credential-socket\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.925109 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.924943 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23cd5145-8ead-40fd-9021-f00dc677eb3b-istiod-ca-cert\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.925109 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.924971 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-data\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.925109 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.924996 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-token\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.925109 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.925032 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-podinfo\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.925109 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.925078 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-workload-socket\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.925109 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.925098 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2jxq\" (UniqueName: \"kubernetes.io/projected/23cd5145-8ead-40fd-9021-f00dc677eb3b-kube-api-access-h2jxq\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.925542 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.925362 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-credential-socket\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.925628 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.925553 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-data\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.925694 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.925636 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-workload-socket\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.925784 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.925763 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-workload-certs\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.925843 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.925807 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23cd5145-8ead-40fd-9021-f00dc677eb3b-istiod-ca-cert\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.927383 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.927365 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-envoy\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.927775 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.927757 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-podinfo\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.934909 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.934881 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-token\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:26.935305 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:26.935282 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2jxq\" (UniqueName: \"kubernetes.io/projected/23cd5145-8ead-40fd-9021-f00dc677eb3b-kube-api-access-h2jxq\") pod \"e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:27.026434 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:27.026333 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:27.182318 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:27.182289 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww"] Jul 21 13:10:27.184950 ip-10-0-133-39 kubenswrapper[2555]: W0721 13:10:27.184920 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23cd5145_8ead_40fd_9021_f00dc677eb3b.slice/crio-e4c17ace83b80059aee62bcdecfc6edfe4cb15fef770a86cd3e5303ab43e9e14 WatchSource:0}: Error finding container e4c17ace83b80059aee62bcdecfc6edfe4cb15fef770a86cd3e5303ab43e9e14: Status 404 returned error can't find the container with id e4c17ace83b80059aee62bcdecfc6edfe4cb15fef770a86cd3e5303ab43e9e14 Jul 21 13:10:27.187287 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:27.187239 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:10:27.187383 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:27.187326 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:10:27.187383 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:27.187356 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:10:27.415073 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:27.415026 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" event={"ID":"23cd5145-8ead-40fd-9021-f00dc677eb3b","Type":"ContainerStarted","Data":"77ccaed0a9f42a1a45486cb7bb6fe0de865aa14719b479847d078dfe0018bee0"} Jul 21 13:10:27.415302 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:27.415102 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" event={"ID":"23cd5145-8ead-40fd-9021-f00dc677eb3b","Type":"ContainerStarted","Data":"e4c17ace83b80059aee62bcdecfc6edfe4cb15fef770a86cd3e5303ab43e9e14"} Jul 21 13:10:27.442498 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:27.442425 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" podStartSLOduration=1.442400341 podStartE2EDuration="1.442400341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:10:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:10:27.439751675 +0000 UTC m=+1850.058044624" watchObservedRunningTime="2026-07-21 13:10:27.442400341 +0000 UTC m=+1850.060693294" Jul 21 13:10:28.026774 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:28.026726 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:28.031941 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:28.031911 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:28.421855 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:28.421818 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:28.422857 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:28.422836 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:28.924384 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:28.924273 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6"] Jul 21 13:10:28.929533 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:28.929484 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" Jul 21 13:10:28.931069 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:28.930479 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6"] Jul 21 13:10:28.933483 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:28.933377 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-njpqk\"" Jul 21 13:10:28.933483 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:28.933384 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 13:10:28.933483 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:28.933444 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-c06656ac-serving-cert\"" Jul 21 13:10:28.934003 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:28.933980 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 13:10:29.042782 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:29.042738 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-maas-api-tls\") pod \"maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6\" (UID: \"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" Jul 21 13:10:29.043187 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:29.042834 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-openshift-service-ca\") pod \"maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6\" (UID: \"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" Jul 21 13:10:29.043187 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:29.042893 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84n48\" (UniqueName: \"kubernetes.io/projected/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-kube-api-access-84n48\") pod \"maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6\" (UID: \"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" Jul 21 13:10:29.143510 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:29.143461 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-openshift-service-ca\") pod \"maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6\" (UID: \"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" Jul 21 13:10:29.143746 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:29.143556 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84n48\" (UniqueName: \"kubernetes.io/projected/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-kube-api-access-84n48\") pod \"maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6\" (UID: \"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" Jul 21 13:10:29.143746 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:29.143626 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-maas-api-tls\") pod \"maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6\" (UID: \"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" Jul 21 13:10:29.143746 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:10:29.143719 2555 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-serving-cert: secret "maas-api-e2e-derive-c06656ac-serving-cert" not found Jul 21 13:10:29.143888 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:10:29.143789 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-maas-api-tls podName:f4226c13-4ba4-4af8-aef9-4d0e52c5c91b nodeName:}" failed. No retries permitted until 2026-07-21 13:10:29.643772085 +0000 UTC m=+1852.262065011 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-maas-api-tls") pod "maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" (UID: "f4226c13-4ba4-4af8-aef9-4d0e52c5c91b") : secret "maas-api-e2e-derive-c06656ac-serving-cert" not found Jul 21 13:10:29.144217 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:29.144195 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-openshift-service-ca\") pod \"maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6\" (UID: \"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" Jul 21 13:10:29.154266 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:29.154239 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84n48\" (UniqueName: \"kubernetes.io/projected/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-kube-api-access-84n48\") pod \"maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6\" (UID: \"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" Jul 21 13:10:29.648420 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:29.648384 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-maas-api-tls\") pod \"maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6\" (UID: \"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" Jul 21 13:10:29.651213 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:29.651184 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-maas-api-tls\") pod \"maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6\" (UID: \"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" Jul 21 13:10:29.842352 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:29.842307 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" Jul 21 13:10:29.991410 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:29.991375 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6"] Jul 21 13:10:29.994613 ip-10-0-133-39 kubenswrapper[2555]: W0721 13:10:29.994564 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4226c13_4ba4_4af8_aef9_4d0e52c5c91b.slice/crio-34d8c495d9c54613bdcae4ff60d0904d53a65c9620b8cdb15d1bcb50dbd45ef7 WatchSource:0}: Error finding container 34d8c495d9c54613bdcae4ff60d0904d53a65c9620b8cdb15d1bcb50dbd45ef7: Status 404 returned error can't find the container with id 34d8c495d9c54613bdcae4ff60d0904d53a65c9620b8cdb15d1bcb50dbd45ef7 Jul 21 13:10:30.435023 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:30.434987 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" event={"ID":"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b","Type":"ContainerStarted","Data":"34d8c495d9c54613bdcae4ff60d0904d53a65c9620b8cdb15d1bcb50dbd45ef7"} Jul 21 13:10:31.442481 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:31.442444 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" event={"ID":"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b","Type":"ContainerStarted","Data":"546df9661d5ca845db19d16b126530d16d189456fbd45941623f3f276129005b"} Jul 21 13:10:31.442936 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:31.442584 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" Jul 21 13:10:31.468269 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:31.468211 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" podStartSLOduration=3.029462217 podStartE2EDuration="3.468195698s" totalImagesPullingTime="438.733481ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:10:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:10:31.466715607 +0000 UTC m=+1854.085008550" watchObservedRunningTime="2026-07-21 13:10:31.468195698 +0000 UTC m=+1854.086488671" Jul 21 13:10:37.453638 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:37.453580 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" Jul 21 13:10:43.423791 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:43.423748 2555 patch_prober.go:28] interesting pod/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 21 13:10:43.424244 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:43.423819 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" podUID="23cd5145-8ead-40fd-9021-f00dc677eb3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 21 13:10:43.508547 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:43.508514 2555 generic.go:356] "Generic (PLEG): container finished" podID="23cd5145-8ead-40fd-9021-f00dc677eb3b" containerID="77ccaed0a9f42a1a45486cb7bb6fe0de865aa14719b479847d078dfe0018bee0" exitCode=0 Jul 21 13:10:43.508739 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:43.508586 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" event={"ID":"23cd5145-8ead-40fd-9021-f00dc677eb3b","Type":"ContainerDied","Data":"77ccaed0a9f42a1a45486cb7bb6fe0de865aa14719b479847d078dfe0018bee0"} Jul 21 13:10:43.509015 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:43.508995 2555 scope.go:117] "RemoveContainer" containerID="77ccaed0a9f42a1a45486cb7bb6fe0de865aa14719b479847d078dfe0018bee0" Jul 21 13:10:43.510003 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:43.509975 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:10:43.510097 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:43.510044 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:10:43.510097 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:43.510074 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:10:44.516811 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:44.516763 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" event={"ID":"23cd5145-8ead-40fd-9021-f00dc677eb3b","Type":"ContainerStarted","Data":"6b66cf0c980f86fb2aa1f6e177e93bd5d318f96e22cd81efcc236e3127644090"} Jul 21 13:10:45.027405 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:45.027370 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:45.027821 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:45.027721 2555 patch_prober.go:28] interesting pod/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 21 13:10:45.027991 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:45.027967 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" podUID="23cd5145-8ead-40fd-9021-f00dc677eb3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 21 13:10:46.027714 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:46.027674 2555 patch_prober.go:28] interesting pod/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 21 13:10:46.028338 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:46.028304 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" podUID="23cd5145-8ead-40fd-9021-f00dc677eb3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 21 13:10:47.027021 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:47.026972 2555 patch_prober.go:28] interesting pod/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 21 13:10:47.027251 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:47.027061 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" podUID="23cd5145-8ead-40fd-9021-f00dc677eb3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 21 13:10:48.026873 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:48.026832 2555 patch_prober.go:28] interesting pod/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 21 13:10:48.027492 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:48.027457 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" podUID="23cd5145-8ead-40fd-9021-f00dc677eb3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 21 13:10:49.030627 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:49.030583 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:49.031090 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:49.031068 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:10:49.031893 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:10:49.031876 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:11:04.248941 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.248905 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6"] Jul 21 13:11:04.249444 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.249127 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" podUID="f4226c13-4ba4-4af8-aef9-4d0e52c5c91b" containerName="maas-api" containerID="cri-o://546df9661d5ca845db19d16b126530d16d189456fbd45941623f3f276129005b" gracePeriod=30 Jul 21 13:11:04.516864 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.516834 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" Jul 21 13:11:04.563783 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.563737 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-openshift-service-ca\") pod \"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b\" (UID: \"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b\") " Jul 21 13:11:04.564006 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.563806 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84n48\" (UniqueName: \"kubernetes.io/projected/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-kube-api-access-84n48\") pod \"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b\" (UID: \"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b\") " Jul 21 13:11:04.564006 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.563848 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-maas-api-tls\") pod \"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b\" (UID: \"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b\") " Jul 21 13:11:04.564177 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.564149 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f4226c13-4ba4-4af8-aef9-4d0e52c5c91b" (UID: "f4226c13-4ba4-4af8-aef9-4d0e52c5c91b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:11:04.566188 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.566161 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f4226c13-4ba4-4af8-aef9-4d0e52c5c91b" (UID: "f4226c13-4ba4-4af8-aef9-4d0e52c5c91b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:11:04.566296 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.566219 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-kube-api-access-84n48" (OuterVolumeSpecName: "kube-api-access-84n48") pod "f4226c13-4ba4-4af8-aef9-4d0e52c5c91b" (UID: "f4226c13-4ba4-4af8-aef9-4d0e52c5c91b"). InnerVolumeSpecName "kube-api-access-84n48". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:11:04.644141 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.644104 2555 generic.go:356] "Generic (PLEG): container finished" podID="f4226c13-4ba4-4af8-aef9-4d0e52c5c91b" containerID="546df9661d5ca845db19d16b126530d16d189456fbd45941623f3f276129005b" exitCode=0 Jul 21 13:11:04.644314 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.644153 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" Jul 21 13:11:04.644314 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.644181 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" event={"ID":"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b","Type":"ContainerDied","Data":"546df9661d5ca845db19d16b126530d16d189456fbd45941623f3f276129005b"} Jul 21 13:11:04.644314 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.644224 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6" event={"ID":"f4226c13-4ba4-4af8-aef9-4d0e52c5c91b","Type":"ContainerDied","Data":"34d8c495d9c54613bdcae4ff60d0904d53a65c9620b8cdb15d1bcb50dbd45ef7"} Jul 21 13:11:04.644314 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.644248 2555 scope.go:117] "RemoveContainer" containerID="546df9661d5ca845db19d16b126530d16d189456fbd45941623f3f276129005b" Jul 21 13:11:04.654647 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.654625 2555 scope.go:117] "RemoveContainer" containerID="546df9661d5ca845db19d16b126530d16d189456fbd45941623f3f276129005b" Jul 21 13:11:04.654944 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:11:04.654925 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"546df9661d5ca845db19d16b126530d16d189456fbd45941623f3f276129005b\": container with ID starting with 546df9661d5ca845db19d16b126530d16d189456fbd45941623f3f276129005b not found: ID does not exist" containerID="546df9661d5ca845db19d16b126530d16d189456fbd45941623f3f276129005b" Jul 21 13:11:04.655021 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.654953 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"546df9661d5ca845db19d16b126530d16d189456fbd45941623f3f276129005b"} err="failed to get container status \"546df9661d5ca845db19d16b126530d16d189456fbd45941623f3f276129005b\": rpc error: code = NotFound desc = could not find container \"546df9661d5ca845db19d16b126530d16d189456fbd45941623f3f276129005b\": container with ID starting with 546df9661d5ca845db19d16b126530d16d189456fbd45941623f3f276129005b not found: ID does not exist" Jul 21 13:11:04.664794 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.664768 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-maas-api-tls\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:11:04.664794 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.664795 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-openshift-service-ca\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:11:04.664794 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.664806 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84n48\" (UniqueName: \"kubernetes.io/projected/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b-kube-api-access-84n48\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:11:04.668025 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.667997 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6"] Jul 21 13:11:04.675459 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:04.675421 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c06656ac-6fc84df995-jfbm6"] Jul 21 13:11:05.927567 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:05.927534 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4226c13-4ba4-4af8-aef9-4d0e52c5c91b" path="/var/lib/kubelet/pods/f4226c13-4ba4-4af8-aef9-4d0e52c5c91b/volumes" Jul 21 13:11:08.925660 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:08.925619 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww"] Jul 21 13:11:08.926117 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:08.925868 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" podUID="23cd5145-8ead-40fd-9021-f00dc677eb3b" containerName="istio-proxy" containerID="cri-o://6b66cf0c980f86fb2aa1f6e177e93bd5d318f96e22cd81efcc236e3127644090" gracePeriod=30 Jul 21 13:11:14.186162 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.186097 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:11:14.241793 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.241743 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-data\") pod \"23cd5145-8ead-40fd-9021-f00dc677eb3b\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " Jul 21 13:11:14.241959 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.241812 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-podinfo\") pod \"23cd5145-8ead-40fd-9021-f00dc677eb3b\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " Jul 21 13:11:14.241959 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.241855 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-workload-socket\") pod \"23cd5145-8ead-40fd-9021-f00dc677eb3b\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " Jul 21 13:11:14.241959 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.241882 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-workload-certs\") pod \"23cd5145-8ead-40fd-9021-f00dc677eb3b\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " Jul 21 13:11:14.241959 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.241918 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23cd5145-8ead-40fd-9021-f00dc677eb3b-istiod-ca-cert\") pod \"23cd5145-8ead-40fd-9021-f00dc677eb3b\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " Jul 21 13:11:14.242204 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.241978 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-token\") pod \"23cd5145-8ead-40fd-9021-f00dc677eb3b\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " Jul 21 13:11:14.242204 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.242040 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-credential-socket\") pod \"23cd5145-8ead-40fd-9021-f00dc677eb3b\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " Jul 21 13:11:14.242204 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.242070 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-envoy\") pod \"23cd5145-8ead-40fd-9021-f00dc677eb3b\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " Jul 21 13:11:14.242204 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.242117 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2jxq\" (UniqueName: \"kubernetes.io/projected/23cd5145-8ead-40fd-9021-f00dc677eb3b-kube-api-access-h2jxq\") pod \"23cd5145-8ead-40fd-9021-f00dc677eb3b\" (UID: \"23cd5145-8ead-40fd-9021-f00dc677eb3b\") " Jul 21 13:11:14.242404 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.242238 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "23cd5145-8ead-40fd-9021-f00dc677eb3b" (UID: "23cd5145-8ead-40fd-9021-f00dc677eb3b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:11:14.242404 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.242310 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "23cd5145-8ead-40fd-9021-f00dc677eb3b" (UID: "23cd5145-8ead-40fd-9021-f00dc677eb3b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:11:14.242505 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.242423 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-workload-certs\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:11:14.242505 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.242443 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-credential-socket\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:11:14.242505 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.242418 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23cd5145-8ead-40fd-9021-f00dc677eb3b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "23cd5145-8ead-40fd-9021-f00dc677eb3b" (UID: "23cd5145-8ead-40fd-9021-f00dc677eb3b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:11:14.242701 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.242583 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "23cd5145-8ead-40fd-9021-f00dc677eb3b" (UID: "23cd5145-8ead-40fd-9021-f00dc677eb3b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:11:14.242760 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.242735 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-data" (OuterVolumeSpecName: "istio-data") pod "23cd5145-8ead-40fd-9021-f00dc677eb3b" (UID: "23cd5145-8ead-40fd-9021-f00dc677eb3b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:11:14.244428 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.244398 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23cd5145-8ead-40fd-9021-f00dc677eb3b-kube-api-access-h2jxq" (OuterVolumeSpecName: "kube-api-access-h2jxq") pod "23cd5145-8ead-40fd-9021-f00dc677eb3b" (UID: "23cd5145-8ead-40fd-9021-f00dc677eb3b"). InnerVolumeSpecName "kube-api-access-h2jxq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:11:14.244428 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.244411 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-token" (OuterVolumeSpecName: "istio-token") pod "23cd5145-8ead-40fd-9021-f00dc677eb3b" (UID: "23cd5145-8ead-40fd-9021-f00dc677eb3b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:11:14.244580 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.244445 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "23cd5145-8ead-40fd-9021-f00dc677eb3b" (UID: "23cd5145-8ead-40fd-9021-f00dc677eb3b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 13:11:14.244646 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.244577 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "23cd5145-8ead-40fd-9021-f00dc677eb3b" (UID: "23cd5145-8ead-40fd-9021-f00dc677eb3b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:11:14.343823 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.343787 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-token\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:11:14.343823 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.343815 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-envoy\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:11:14.343823 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.343827 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2jxq\" (UniqueName: \"kubernetes.io/projected/23cd5145-8ead-40fd-9021-f00dc677eb3b-kube-api-access-h2jxq\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:11:14.344063 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.343835 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-data\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:11:14.344063 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.343848 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23cd5145-8ead-40fd-9021-f00dc677eb3b-istio-podinfo\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:11:14.344063 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.343859 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23cd5145-8ead-40fd-9021-f00dc677eb3b-workload-socket\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:11:14.344063 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.343866 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23cd5145-8ead-40fd-9021-f00dc677eb3b-istiod-ca-cert\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:11:14.700350 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.700316 2555 generic.go:356] "Generic (PLEG): container finished" podID="23cd5145-8ead-40fd-9021-f00dc677eb3b" containerID="6b66cf0c980f86fb2aa1f6e177e93bd5d318f96e22cd81efcc236e3127644090" exitCode=0 Jul 21 13:11:14.700548 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.700364 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" Jul 21 13:11:14.700548 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.700401 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" event={"ID":"23cd5145-8ead-40fd-9021-f00dc677eb3b","Type":"ContainerDied","Data":"6b66cf0c980f86fb2aa1f6e177e93bd5d318f96e22cd81efcc236e3127644090"} Jul 21 13:11:14.700548 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.700459 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww" event={"ID":"23cd5145-8ead-40fd-9021-f00dc677eb3b","Type":"ContainerDied","Data":"e4c17ace83b80059aee62bcdecfc6edfe4cb15fef770a86cd3e5303ab43e9e14"} Jul 21 13:11:14.700548 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.700481 2555 scope.go:117] "RemoveContainer" containerID="6b66cf0c980f86fb2aa1f6e177e93bd5d318f96e22cd81efcc236e3127644090" Jul 21 13:11:14.710679 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.710656 2555 scope.go:117] "RemoveContainer" containerID="77ccaed0a9f42a1a45486cb7bb6fe0de865aa14719b479847d078dfe0018bee0" Jul 21 13:11:14.720862 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.720772 2555 scope.go:117] "RemoveContainer" containerID="6b66cf0c980f86fb2aa1f6e177e93bd5d318f96e22cd81efcc236e3127644090" Jul 21 13:11:14.721490 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:11:14.721280 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b66cf0c980f86fb2aa1f6e177e93bd5d318f96e22cd81efcc236e3127644090\": container with ID starting with 6b66cf0c980f86fb2aa1f6e177e93bd5d318f96e22cd81efcc236e3127644090 not found: ID does not exist" containerID="6b66cf0c980f86fb2aa1f6e177e93bd5d318f96e22cd81efcc236e3127644090" Jul 21 13:11:14.721490 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.721332 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b66cf0c980f86fb2aa1f6e177e93bd5d318f96e22cd81efcc236e3127644090"} err="failed to get container status \"6b66cf0c980f86fb2aa1f6e177e93bd5d318f96e22cd81efcc236e3127644090\": rpc error: code = NotFound desc = could not find container \"6b66cf0c980f86fb2aa1f6e177e93bd5d318f96e22cd81efcc236e3127644090\": container with ID starting with 6b66cf0c980f86fb2aa1f6e177e93bd5d318f96e22cd81efcc236e3127644090 not found: ID does not exist" Jul 21 13:11:14.721490 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.721359 2555 scope.go:117] "RemoveContainer" containerID="77ccaed0a9f42a1a45486cb7bb6fe0de865aa14719b479847d078dfe0018bee0" Jul 21 13:11:14.722157 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:11:14.722074 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77ccaed0a9f42a1a45486cb7bb6fe0de865aa14719b479847d078dfe0018bee0\": container with ID starting with 77ccaed0a9f42a1a45486cb7bb6fe0de865aa14719b479847d078dfe0018bee0 not found: ID does not exist" containerID="77ccaed0a9f42a1a45486cb7bb6fe0de865aa14719b479847d078dfe0018bee0" Jul 21 13:11:14.722157 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.722100 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77ccaed0a9f42a1a45486cb7bb6fe0de865aa14719b479847d078dfe0018bee0"} err="failed to get container status \"77ccaed0a9f42a1a45486cb7bb6fe0de865aa14719b479847d078dfe0018bee0\": rpc error: code = NotFound desc = could not find container \"77ccaed0a9f42a1a45486cb7bb6fe0de865aa14719b479847d078dfe0018bee0\": container with ID starting with 77ccaed0a9f42a1a45486cb7bb6fe0de865aa14719b479847d078dfe0018bee0 not found: ID does not exist" Jul 21 13:11:14.725535 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.725508 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww"] Jul 21 13:11:14.731435 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:14.731414 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-c06656ac-openshift-default-54c58b795f-tzbww"] Jul 21 13:11:15.925934 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:11:15.925900 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23cd5145-8ead-40fd-9021-f00dc677eb3b" path="/var/lib/kubelet/pods/23cd5145-8ead-40fd-9021-f00dc677eb3b/volumes" Jul 21 13:12:56.846634 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.846525 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz"] Jul 21 13:12:56.847111 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.846912 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23cd5145-8ead-40fd-9021-f00dc677eb3b" containerName="istio-proxy" Jul 21 13:12:56.847111 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.846923 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="23cd5145-8ead-40fd-9021-f00dc677eb3b" containerName="istio-proxy" Jul 21 13:12:56.847111 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.846935 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23cd5145-8ead-40fd-9021-f00dc677eb3b" containerName="istio-proxy" Jul 21 13:12:56.847111 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.846940 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="23cd5145-8ead-40fd-9021-f00dc677eb3b" containerName="istio-proxy" Jul 21 13:12:56.847111 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.846958 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4226c13-4ba4-4af8-aef9-4d0e52c5c91b" containerName="maas-api" Jul 21 13:12:56.847111 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.846963 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4226c13-4ba4-4af8-aef9-4d0e52c5c91b" containerName="maas-api" Jul 21 13:12:56.847111 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.847019 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="23cd5145-8ead-40fd-9021-f00dc677eb3b" containerName="istio-proxy" Jul 21 13:12:56.847111 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.847028 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4226c13-4ba4-4af8-aef9-4d0e52c5c91b" containerName="maas-api" Jul 21 13:12:56.850115 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.850084 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:56.853737 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.853713 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4de9b363-openshift-default-dockercfg-49nzz\"" Jul 21 13:12:56.881424 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.881388 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz"] Jul 21 13:12:56.968032 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.968000 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6s4r\" (UniqueName: \"kubernetes.io/projected/60cc6c59-f7c3-4d90-830a-818d2e99c42a-kube-api-access-f6s4r\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:56.968224 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.968136 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-podinfo\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:56.968224 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.968161 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istiod-ca-cert\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:56.968316 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.968242 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-credential-socket\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:56.968316 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.968262 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-token\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:56.968316 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.968290 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-data\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:56.968455 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.968383 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-workload-socket\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:56.968455 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.968438 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-workload-certs\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:56.968544 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:56.968464 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-envoy\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.069769 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.069731 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-workload-socket\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.069769 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.069771 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-workload-certs\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.070037 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.069787 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-envoy\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.070037 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.069817 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6s4r\" (UniqueName: \"kubernetes.io/projected/60cc6c59-f7c3-4d90-830a-818d2e99c42a-kube-api-access-f6s4r\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.070037 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.069891 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-podinfo\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.070037 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.069916 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istiod-ca-cert\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.070037 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.069955 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-credential-socket\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.070037 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.069974 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-token\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.070037 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.070000 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-data\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.070387 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.070184 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-workload-socket\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.070387 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.070263 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-workload-certs\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.070518 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.070493 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-credential-socket\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.070650 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.070633 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-data\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.071029 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.071005 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istiod-ca-cert\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.072455 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.072432 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-envoy\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.072564 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.072535 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-podinfo\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.087117 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.087088 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6s4r\" (UniqueName: \"kubernetes.io/projected/60cc6c59-f7c3-4d90-830a-818d2e99c42a-kube-api-access-f6s4r\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.087623 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.087572 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-token\") pod \"e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.161627 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.161499 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:57.313640 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.313614 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz"] Jul 21 13:12:57.316117 ip-10-0-133-39 kubenswrapper[2555]: W0721 13:12:57.316089 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60cc6c59_f7c3_4d90_830a_818d2e99c42a.slice/crio-40d7013e87295f59fbd2294cc084713fbd0373b44b4a776a4191cacb9d077b45 WatchSource:0}: Error finding container 40d7013e87295f59fbd2294cc084713fbd0373b44b4a776a4191cacb9d077b45: Status 404 returned error can't find the container with id 40d7013e87295f59fbd2294cc084713fbd0373b44b4a776a4191cacb9d077b45 Jul 21 13:12:57.318283 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.318251 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:12:57.318370 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.318322 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:12:57.318370 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:57.318356 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:12:58.249348 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:58.249315 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" event={"ID":"60cc6c59-f7c3-4d90-830a-818d2e99c42a","Type":"ContainerStarted","Data":"db07988240fb948c7657f0d5a570c5fb362a90ba9b1a2e953c9e412501385c49"} Jul 21 13:12:58.249770 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:58.249367 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" event={"ID":"60cc6c59-f7c3-4d90-830a-818d2e99c42a","Type":"ContainerStarted","Data":"40d7013e87295f59fbd2294cc084713fbd0373b44b4a776a4191cacb9d077b45"} Jul 21 13:12:58.271727 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:58.271666 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" podStartSLOduration=2.271644611 podStartE2EDuration="2.271644611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:12:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:12:58.27056458 +0000 UTC m=+2000.888857531" watchObservedRunningTime="2026-07-21 13:12:58.271644611 +0000 UTC m=+2000.889937562" Jul 21 13:12:59.162161 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:59.162118 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:59.167453 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:59.167427 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:59.254334 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:59.254300 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:12:59.255259 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:12:59.255239 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:13:00.389299 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:00.389257 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq"] Jul 21 13:13:00.389983 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:00.389957 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="23cd5145-8ead-40fd-9021-f00dc677eb3b" containerName="istio-proxy" Jul 21 13:13:00.393524 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:00.393501 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" Jul 21 13:13:00.395910 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:00.395886 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-njpqk\"" Jul 21 13:13:00.396071 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:00.395943 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4de9b363-serving-cert\"" Jul 21 13:13:00.396149 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:00.395999 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 13:13:00.396562 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:00.396541 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 13:13:00.403558 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:00.403529 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq"] Jul 21 13:13:00.505659 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:00.505587 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2shhm\" (UniqueName: \"kubernetes.io/projected/a773b2fc-7561-44ee-9664-43c5bdc2d4b6-kube-api-access-2shhm\") pod \"maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq\" (UID: \"a773b2fc-7561-44ee-9664-43c5bdc2d4b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" Jul 21 13:13:00.505849 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:00.505682 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a773b2fc-7561-44ee-9664-43c5bdc2d4b6-openshift-service-ca\") pod \"maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq\" (UID: \"a773b2fc-7561-44ee-9664-43c5bdc2d4b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" Jul 21 13:13:00.505849 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:00.505715 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a773b2fc-7561-44ee-9664-43c5bdc2d4b6-maas-api-tls\") pod \"maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq\" (UID: \"a773b2fc-7561-44ee-9664-43c5bdc2d4b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" Jul 21 13:13:00.606464 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:00.606422 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2shhm\" (UniqueName: \"kubernetes.io/projected/a773b2fc-7561-44ee-9664-43c5bdc2d4b6-kube-api-access-2shhm\") pod \"maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq\" (UID: \"a773b2fc-7561-44ee-9664-43c5bdc2d4b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" Jul 21 13:13:00.606708 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:00.606486 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a773b2fc-7561-44ee-9664-43c5bdc2d4b6-openshift-service-ca\") pod \"maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq\" (UID: \"a773b2fc-7561-44ee-9664-43c5bdc2d4b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" Jul 21 13:13:00.606708 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:00.606534 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a773b2fc-7561-44ee-9664-43c5bdc2d4b6-maas-api-tls\") pod \"maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq\" (UID: \"a773b2fc-7561-44ee-9664-43c5bdc2d4b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" Jul 21 13:13:00.607232 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:00.607205 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a773b2fc-7561-44ee-9664-43c5bdc2d4b6-openshift-service-ca\") pod \"maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq\" (UID: \"a773b2fc-7561-44ee-9664-43c5bdc2d4b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" Jul 21 13:13:00.609348 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:00.609324 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a773b2fc-7561-44ee-9664-43c5bdc2d4b6-maas-api-tls\") pod \"maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq\" (UID: \"a773b2fc-7561-44ee-9664-43c5bdc2d4b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" Jul 21 13:13:00.615404 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:00.615375 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2shhm\" (UniqueName: \"kubernetes.io/projected/a773b2fc-7561-44ee-9664-43c5bdc2d4b6-kube-api-access-2shhm\") pod \"maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq\" (UID: \"a773b2fc-7561-44ee-9664-43c5bdc2d4b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" Jul 21 13:13:00.707878 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:00.707771 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" Jul 21 13:13:00.855462 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:00.855437 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq"] Jul 21 13:13:00.858062 ip-10-0-133-39 kubenswrapper[2555]: W0721 13:13:00.858024 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda773b2fc_7561_44ee_9664_43c5bdc2d4b6.slice/crio-42d356ceebcfdeba5c86af30c92e5c989dc37d3d67908ddc3d2ad470f1c13f87 WatchSource:0}: Error finding container 42d356ceebcfdeba5c86af30c92e5c989dc37d3d67908ddc3d2ad470f1c13f87: Status 404 returned error can't find the container with id 42d356ceebcfdeba5c86af30c92e5c989dc37d3d67908ddc3d2ad470f1c13f87 Jul 21 13:13:01.266387 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:01.266345 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" event={"ID":"a773b2fc-7561-44ee-9664-43c5bdc2d4b6","Type":"ContainerStarted","Data":"42d356ceebcfdeba5c86af30c92e5c989dc37d3d67908ddc3d2ad470f1c13f87"} Jul 21 13:13:02.273256 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:02.273220 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" event={"ID":"a773b2fc-7561-44ee-9664-43c5bdc2d4b6","Type":"ContainerStarted","Data":"cfb012df6980bd506895e179d48416f8137cae19f8dd619d9d0f761e225e7a4a"} Jul 21 13:13:02.273758 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:02.273284 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" Jul 21 13:13:02.292225 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:02.292174 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" podStartSLOduration=1.827517393 podStartE2EDuration="2.292159159s" totalImagesPullingTime="464.641766ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:13:02.290052589 +0000 UTC m=+2004.908345534" watchObservedRunningTime="2026-07-21 13:13:02.292159159 +0000 UTC m=+2004.910452380" Jul 21 13:13:08.285111 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:08.285080 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" Jul 21 13:13:14.256236 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:14.256194 2555 patch_prober.go:28] interesting pod/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 21 13:13:14.256740 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:14.256272 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" podUID="60cc6c59-f7c3-4d90-830a-818d2e99c42a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 21 13:13:14.338925 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:14.338888 2555 generic.go:356] "Generic (PLEG): container finished" podID="60cc6c59-f7c3-4d90-830a-818d2e99c42a" containerID="db07988240fb948c7657f0d5a570c5fb362a90ba9b1a2e953c9e412501385c49" exitCode=0 Jul 21 13:13:14.339118 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:14.338977 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" event={"ID":"60cc6c59-f7c3-4d90-830a-818d2e99c42a","Type":"ContainerDied","Data":"db07988240fb948c7657f0d5a570c5fb362a90ba9b1a2e953c9e412501385c49"} Jul 21 13:13:14.339423 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:14.339408 2555 scope.go:117] "RemoveContainer" containerID="db07988240fb948c7657f0d5a570c5fb362a90ba9b1a2e953c9e412501385c49" Jul 21 13:13:14.340509 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:14.340483 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:13:14.340568 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:14.340549 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:13:14.340632 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:14.340582 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:13:15.348239 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:15.348206 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" event={"ID":"60cc6c59-f7c3-4d90-830a-818d2e99c42a","Type":"ContainerStarted","Data":"437cf7892d21760545aebac79eb894842d1fbb8dd3106ac09aa194a1afc1a07a"} Jul 21 13:13:16.162511 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:16.162474 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:13:16.162791 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:16.162767 2555 patch_prober.go:28] interesting pod/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 21 13:13:16.162904 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:16.162822 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" podUID="60cc6c59-f7c3-4d90-830a-818d2e99c42a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 21 13:13:17.161961 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:17.161927 2555 patch_prober.go:28] interesting pod/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 21 13:13:17.162369 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:17.161994 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" podUID="60cc6c59-f7c3-4d90-830a-818d2e99c42a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 21 13:13:18.162011 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:18.161971 2555 patch_prober.go:28] interesting pod/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 21 13:13:18.162387 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:18.162039 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" podUID="60cc6c59-f7c3-4d90-830a-818d2e99c42a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 21 13:13:19.162055 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:19.162012 2555 patch_prober.go:28] interesting pod/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 21 13:13:19.162461 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:19.162086 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" podUID="60cc6c59-f7c3-4d90-830a-818d2e99c42a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 21 13:13:20.166989 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:20.166955 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:13:20.167379 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:20.167273 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:13:20.168248 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:20.168225 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:13:23.502634 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:23.502585 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq"] Jul 21 13:13:23.503100 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:23.502829 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" podUID="a773b2fc-7561-44ee-9664-43c5bdc2d4b6" containerName="maas-api" containerID="cri-o://cfb012df6980bd506895e179d48416f8137cae19f8dd619d9d0f761e225e7a4a" gracePeriod=30 Jul 21 13:13:23.766903 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:23.766822 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" Jul 21 13:13:23.822217 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:23.822184 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a773b2fc-7561-44ee-9664-43c5bdc2d4b6-openshift-service-ca\") pod \"a773b2fc-7561-44ee-9664-43c5bdc2d4b6\" (UID: \"a773b2fc-7561-44ee-9664-43c5bdc2d4b6\") " Jul 21 13:13:23.822217 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:23.822231 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2shhm\" (UniqueName: \"kubernetes.io/projected/a773b2fc-7561-44ee-9664-43c5bdc2d4b6-kube-api-access-2shhm\") pod \"a773b2fc-7561-44ee-9664-43c5bdc2d4b6\" (UID: \"a773b2fc-7561-44ee-9664-43c5bdc2d4b6\") " Jul 21 13:13:23.822488 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:23.822287 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a773b2fc-7561-44ee-9664-43c5bdc2d4b6-maas-api-tls\") pod \"a773b2fc-7561-44ee-9664-43c5bdc2d4b6\" (UID: \"a773b2fc-7561-44ee-9664-43c5bdc2d4b6\") " Jul 21 13:13:23.822662 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:23.822624 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a773b2fc-7561-44ee-9664-43c5bdc2d4b6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a773b2fc-7561-44ee-9664-43c5bdc2d4b6" (UID: "a773b2fc-7561-44ee-9664-43c5bdc2d4b6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:13:23.824651 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:23.824622 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a773b2fc-7561-44ee-9664-43c5bdc2d4b6-kube-api-access-2shhm" (OuterVolumeSpecName: "kube-api-access-2shhm") pod "a773b2fc-7561-44ee-9664-43c5bdc2d4b6" (UID: "a773b2fc-7561-44ee-9664-43c5bdc2d4b6"). InnerVolumeSpecName "kube-api-access-2shhm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:13:23.824748 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:23.824628 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a773b2fc-7561-44ee-9664-43c5bdc2d4b6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a773b2fc-7561-44ee-9664-43c5bdc2d4b6" (UID: "a773b2fc-7561-44ee-9664-43c5bdc2d4b6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:13:23.923779 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:23.923752 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a773b2fc-7561-44ee-9664-43c5bdc2d4b6-maas-api-tls\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:13:23.923779 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:23.923776 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a773b2fc-7561-44ee-9664-43c5bdc2d4b6-openshift-service-ca\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:13:23.923779 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:23.923785 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2shhm\" (UniqueName: \"kubernetes.io/projected/a773b2fc-7561-44ee-9664-43c5bdc2d4b6-kube-api-access-2shhm\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:13:24.398361 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:24.398325 2555 generic.go:356] "Generic (PLEG): container finished" podID="a773b2fc-7561-44ee-9664-43c5bdc2d4b6" containerID="cfb012df6980bd506895e179d48416f8137cae19f8dd619d9d0f761e225e7a4a" exitCode=0 Jul 21 13:13:24.398548 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:24.398367 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" Jul 21 13:13:24.398548 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:24.398407 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" event={"ID":"a773b2fc-7561-44ee-9664-43c5bdc2d4b6","Type":"ContainerDied","Data":"cfb012df6980bd506895e179d48416f8137cae19f8dd619d9d0f761e225e7a4a"} Jul 21 13:13:24.398548 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:24.398461 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq" event={"ID":"a773b2fc-7561-44ee-9664-43c5bdc2d4b6","Type":"ContainerDied","Data":"42d356ceebcfdeba5c86af30c92e5c989dc37d3d67908ddc3d2ad470f1c13f87"} Jul 21 13:13:24.398548 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:24.398483 2555 scope.go:117] "RemoveContainer" containerID="cfb012df6980bd506895e179d48416f8137cae19f8dd619d9d0f761e225e7a4a" Jul 21 13:13:24.407694 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:24.407675 2555 scope.go:117] "RemoveContainer" containerID="cfb012df6980bd506895e179d48416f8137cae19f8dd619d9d0f761e225e7a4a" Jul 21 13:13:24.408023 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:13:24.408002 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfb012df6980bd506895e179d48416f8137cae19f8dd619d9d0f761e225e7a4a\": container with ID starting with cfb012df6980bd506895e179d48416f8137cae19f8dd619d9d0f761e225e7a4a not found: ID does not exist" containerID="cfb012df6980bd506895e179d48416f8137cae19f8dd619d9d0f761e225e7a4a" Jul 21 13:13:24.408123 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:24.408033 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfb012df6980bd506895e179d48416f8137cae19f8dd619d9d0f761e225e7a4a"} err="failed to get container status \"cfb012df6980bd506895e179d48416f8137cae19f8dd619d9d0f761e225e7a4a\": rpc error: code = NotFound desc = could not find container \"cfb012df6980bd506895e179d48416f8137cae19f8dd619d9d0f761e225e7a4a\": container with ID starting with cfb012df6980bd506895e179d48416f8137cae19f8dd619d9d0f761e225e7a4a not found: ID does not exist" Jul 21 13:13:24.426945 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:24.426898 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq"] Jul 21 13:13:24.427135 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:24.426965 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4de9b363-6f785ccd79-jk6gq"] Jul 21 13:13:25.926068 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:25.926032 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a773b2fc-7561-44ee-9664-43c5bdc2d4b6" path="/var/lib/kubelet/pods/a773b2fc-7561-44ee-9664-43c5bdc2d4b6/volumes" Jul 21 13:13:35.023108 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:35.023066 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz"] Jul 21 13:13:35.023751 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:35.023278 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" podUID="60cc6c59-f7c3-4d90-830a-818d2e99c42a" containerName="istio-proxy" containerID="cri-o://437cf7892d21760545aebac79eb894842d1fbb8dd3106ac09aa194a1afc1a07a" gracePeriod=30 Jul 21 13:13:35.169667 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:35.169623 2555 patch_prober.go:28] interesting pod/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 21 13:13:35.169858 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:35.169704 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" podUID="60cc6c59-f7c3-4d90-830a-818d2e99c42a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 21 13:13:40.289610 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.289531 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:13:40.367536 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.367493 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-envoy\") pod \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " Jul 21 13:13:40.367733 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.367560 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-token\") pod \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " Jul 21 13:13:40.367733 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.367617 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-podinfo\") pod \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " Jul 21 13:13:40.367733 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.367700 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istiod-ca-cert\") pod \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " Jul 21 13:13:40.367733 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.367722 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-credential-socket\") pod \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " Jul 21 13:13:40.367974 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.367746 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-data\") pod \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " Jul 21 13:13:40.367974 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.367766 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-workload-certs\") pod \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " Jul 21 13:13:40.367974 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.367808 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-workload-socket\") pod \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " Jul 21 13:13:40.367974 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.367871 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6s4r\" (UniqueName: \"kubernetes.io/projected/60cc6c59-f7c3-4d90-830a-818d2e99c42a-kube-api-access-f6s4r\") pod \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\" (UID: \"60cc6c59-f7c3-4d90-830a-818d2e99c42a\") " Jul 21 13:13:40.368163 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.368020 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "60cc6c59-f7c3-4d90-830a-818d2e99c42a" (UID: "60cc6c59-f7c3-4d90-830a-818d2e99c42a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:13:40.368222 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.368202 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-workload-certs\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:13:40.368331 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.368306 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "60cc6c59-f7c3-4d90-830a-818d2e99c42a" (UID: "60cc6c59-f7c3-4d90-830a-818d2e99c42a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:13:40.368439 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.368333 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "60cc6c59-f7c3-4d90-830a-818d2e99c42a" (UID: "60cc6c59-f7c3-4d90-830a-818d2e99c42a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:13:40.368726 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.368702 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "60cc6c59-f7c3-4d90-830a-818d2e99c42a" (UID: "60cc6c59-f7c3-4d90-830a-818d2e99c42a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:13:40.368726 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.368707 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-data" (OuterVolumeSpecName: "istio-data") pod "60cc6c59-f7c3-4d90-830a-818d2e99c42a" (UID: "60cc6c59-f7c3-4d90-830a-818d2e99c42a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:13:40.370226 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.370197 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60cc6c59-f7c3-4d90-830a-818d2e99c42a-kube-api-access-f6s4r" (OuterVolumeSpecName: "kube-api-access-f6s4r") pod "60cc6c59-f7c3-4d90-830a-818d2e99c42a" (UID: "60cc6c59-f7c3-4d90-830a-818d2e99c42a"). InnerVolumeSpecName "kube-api-access-f6s4r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:13:40.370226 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.370218 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "60cc6c59-f7c3-4d90-830a-818d2e99c42a" (UID: "60cc6c59-f7c3-4d90-830a-818d2e99c42a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 13:13:40.370394 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.370378 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "60cc6c59-f7c3-4d90-830a-818d2e99c42a" (UID: "60cc6c59-f7c3-4d90-830a-818d2e99c42a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:13:40.370435 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.370401 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-token" (OuterVolumeSpecName: "istio-token") pod "60cc6c59-f7c3-4d90-830a-818d2e99c42a" (UID: "60cc6c59-f7c3-4d90-830a-818d2e99c42a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:13:40.468886 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.468848 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-workload-socket\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:13:40.468886 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.468879 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6s4r\" (UniqueName: \"kubernetes.io/projected/60cc6c59-f7c3-4d90-830a-818d2e99c42a-kube-api-access-f6s4r\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:13:40.468886 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.468888 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-envoy\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:13:40.468886 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.468897 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-token\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:13:40.468886 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.468904 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-podinfo\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:13:40.469198 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.468913 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istiod-ca-cert\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:13:40.469198 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.468921 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-credential-socket\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:13:40.469198 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.468928 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60cc6c59-f7c3-4d90-830a-818d2e99c42a-istio-data\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:13:40.489994 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.489961 2555 generic.go:356] "Generic (PLEG): container finished" podID="60cc6c59-f7c3-4d90-830a-818d2e99c42a" containerID="437cf7892d21760545aebac79eb894842d1fbb8dd3106ac09aa194a1afc1a07a" exitCode=0 Jul 21 13:13:40.490132 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.490019 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" Jul 21 13:13:40.490132 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.490048 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" event={"ID":"60cc6c59-f7c3-4d90-830a-818d2e99c42a","Type":"ContainerDied","Data":"437cf7892d21760545aebac79eb894842d1fbb8dd3106ac09aa194a1afc1a07a"} Jul 21 13:13:40.490132 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.490110 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz" event={"ID":"60cc6c59-f7c3-4d90-830a-818d2e99c42a","Type":"ContainerDied","Data":"40d7013e87295f59fbd2294cc084713fbd0373b44b4a776a4191cacb9d077b45"} Jul 21 13:13:40.490272 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.490140 2555 scope.go:117] "RemoveContainer" containerID="437cf7892d21760545aebac79eb894842d1fbb8dd3106ac09aa194a1afc1a07a" Jul 21 13:13:40.500638 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.500587 2555 scope.go:117] "RemoveContainer" containerID="db07988240fb948c7657f0d5a570c5fb362a90ba9b1a2e953c9e412501385c49" Jul 21 13:13:40.509954 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.509936 2555 scope.go:117] "RemoveContainer" containerID="437cf7892d21760545aebac79eb894842d1fbb8dd3106ac09aa194a1afc1a07a" Jul 21 13:13:40.510255 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:13:40.510234 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"437cf7892d21760545aebac79eb894842d1fbb8dd3106ac09aa194a1afc1a07a\": container with ID starting with 437cf7892d21760545aebac79eb894842d1fbb8dd3106ac09aa194a1afc1a07a not found: ID does not exist" containerID="437cf7892d21760545aebac79eb894842d1fbb8dd3106ac09aa194a1afc1a07a" Jul 21 13:13:40.510304 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.510264 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"437cf7892d21760545aebac79eb894842d1fbb8dd3106ac09aa194a1afc1a07a"} err="failed to get container status \"437cf7892d21760545aebac79eb894842d1fbb8dd3106ac09aa194a1afc1a07a\": rpc error: code = NotFound desc = could not find container \"437cf7892d21760545aebac79eb894842d1fbb8dd3106ac09aa194a1afc1a07a\": container with ID starting with 437cf7892d21760545aebac79eb894842d1fbb8dd3106ac09aa194a1afc1a07a not found: ID does not exist" Jul 21 13:13:40.510304 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.510280 2555 scope.go:117] "RemoveContainer" containerID="db07988240fb948c7657f0d5a570c5fb362a90ba9b1a2e953c9e412501385c49" Jul 21 13:13:40.510544 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:13:40.510523 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db07988240fb948c7657f0d5a570c5fb362a90ba9b1a2e953c9e412501385c49\": container with ID starting with db07988240fb948c7657f0d5a570c5fb362a90ba9b1a2e953c9e412501385c49 not found: ID does not exist" containerID="db07988240fb948c7657f0d5a570c5fb362a90ba9b1a2e953c9e412501385c49" Jul 21 13:13:40.510641 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.510553 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db07988240fb948c7657f0d5a570c5fb362a90ba9b1a2e953c9e412501385c49"} err="failed to get container status \"db07988240fb948c7657f0d5a570c5fb362a90ba9b1a2e953c9e412501385c49\": rpc error: code = NotFound desc = could not find container \"db07988240fb948c7657f0d5a570c5fb362a90ba9b1a2e953c9e412501385c49\": container with ID starting with db07988240fb948c7657f0d5a570c5fb362a90ba9b1a2e953c9e412501385c49 not found: ID does not exist" Jul 21 13:13:40.523706 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.523675 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz"] Jul 21 13:13:40.529511 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:40.529484 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4de9b363-openshift-default-55dd94d4d7-zf2qz"] Jul 21 13:13:41.926254 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:41.926219 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60cc6c59-f7c3-4d90-830a-818d2e99c42a" path="/var/lib/kubelet/pods/60cc6c59-f7c3-4d90-830a-818d2e99c42a/volumes" Jul 21 13:13:55.726736 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.726700 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph"] Jul 21 13:13:55.727116 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.727048 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60cc6c59-f7c3-4d90-830a-818d2e99c42a" containerName="istio-proxy" Jul 21 13:13:55.727116 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.727059 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="60cc6c59-f7c3-4d90-830a-818d2e99c42a" containerName="istio-proxy" Jul 21 13:13:55.727116 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.727070 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a773b2fc-7561-44ee-9664-43c5bdc2d4b6" containerName="maas-api" Jul 21 13:13:55.727116 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.727075 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a773b2fc-7561-44ee-9664-43c5bdc2d4b6" containerName="maas-api" Jul 21 13:13:55.727242 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.727132 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="60cc6c59-f7c3-4d90-830a-818d2e99c42a" containerName="istio-proxy" Jul 21 13:13:55.727242 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.727141 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="60cc6c59-f7c3-4d90-830a-818d2e99c42a" containerName="istio-proxy" Jul 21 13:13:55.727242 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.727152 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a773b2fc-7561-44ee-9664-43c5bdc2d4b6" containerName="maas-api" Jul 21 13:13:55.731447 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.731429 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.733901 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.733881 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4f0dda7a-openshift-default-dockercfg-s8mlj\"" Jul 21 13:13:55.744860 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.744835 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph"] Jul 21 13:13:55.799447 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.799406 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-credential-socket\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.799447 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.799450 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-podinfo\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.799704 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.799473 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-envoy\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.799704 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.799526 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-data\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.799704 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.799561 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-workload-certs\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.799704 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.799623 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-token\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.799704 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.799680 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlncc\" (UniqueName: \"kubernetes.io/projected/5ac73392-1a91-4fea-975f-2d145ab20a7c-kube-api-access-rlncc\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.799926 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.799725 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-workload-socket\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.799926 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.799753 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ac73392-1a91-4fea-975f-2d145ab20a7c-istiod-ca-cert\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.901111 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.901073 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-token\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.901312 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.901128 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlncc\" (UniqueName: \"kubernetes.io/projected/5ac73392-1a91-4fea-975f-2d145ab20a7c-kube-api-access-rlncc\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.901312 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.901148 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-workload-socket\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.901312 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.901167 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ac73392-1a91-4fea-975f-2d145ab20a7c-istiod-ca-cert\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.901312 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.901194 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-credential-socket\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.901312 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.901222 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-podinfo\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.901312 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.901254 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-envoy\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.901312 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.901282 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-data\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.901312 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.901307 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-workload-certs\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.901783 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.901664 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-workload-socket\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.901783 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.901716 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-workload-certs\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.901900 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.901847 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-credential-socket\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.901900 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.901858 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-data\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.902071 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.902052 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ac73392-1a91-4fea-975f-2d145ab20a7c-istiod-ca-cert\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.903891 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.903870 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-envoy\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.904094 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.904079 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-podinfo\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.913799 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.913774 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-token\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:55.919800 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:55.919763 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlncc\" (UniqueName: \"kubernetes.io/projected/5ac73392-1a91-4fea-975f-2d145ab20a7c-kube-api-access-rlncc\") pod \"e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:56.042834 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:56.042735 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:56.186369 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:56.186344 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph"] Jul 21 13:13:56.188750 ip-10-0-133-39 kubenswrapper[2555]: W0721 13:13:56.188722 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ac73392_1a91_4fea_975f_2d145ab20a7c.slice/crio-d38f7d47362ca68b42e4acffbf7eac012b5c4999133e39aaeac1a4cfade8e7d6 WatchSource:0}: Error finding container d38f7d47362ca68b42e4acffbf7eac012b5c4999133e39aaeac1a4cfade8e7d6: Status 404 returned error can't find the container with id d38f7d47362ca68b42e4acffbf7eac012b5c4999133e39aaeac1a4cfade8e7d6 Jul 21 13:13:56.190890 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:56.190858 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:13:56.190959 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:56.190925 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:13:56.190959 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:56.190954 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:13:56.574069 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:56.574037 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" event={"ID":"5ac73392-1a91-4fea-975f-2d145ab20a7c","Type":"ContainerStarted","Data":"d1d5a83119478f2f530aa6b776c2731ca5f23b1178ef1caa38f517ed098d8842"} Jul 21 13:13:56.574253 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:56.574085 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" event={"ID":"5ac73392-1a91-4fea-975f-2d145ab20a7c","Type":"ContainerStarted","Data":"d38f7d47362ca68b42e4acffbf7eac012b5c4999133e39aaeac1a4cfade8e7d6"} Jul 21 13:13:56.596036 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:56.595985 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" podStartSLOduration=1.5959701640000001 podStartE2EDuration="1.595970164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:13:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:13:56.594229286 +0000 UTC m=+2059.212522234" watchObservedRunningTime="2026-07-21 13:13:56.595970164 +0000 UTC m=+2059.214263111" Jul 21 13:13:57.043571 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:57.043534 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:57.048928 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:57.048896 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:57.579440 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:57.579416 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:57.580714 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:57.580698 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:13:58.645008 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:58.644929 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq"] Jul 21 13:13:58.645384 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:58.645314 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60cc6c59-f7c3-4d90-830a-818d2e99c42a" containerName="istio-proxy" Jul 21 13:13:58.645384 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:58.645325 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="60cc6c59-f7c3-4d90-830a-818d2e99c42a" containerName="istio-proxy" Jul 21 13:13:58.648665 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:58.648642 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" Jul 21 13:13:58.650891 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:58.650869 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 13:13:58.651461 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:58.651446 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 13:13:58.651510 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:58.651471 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4f0dda7a-serving-cert\"" Jul 21 13:13:58.651550 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:58.651445 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-njpqk\"" Jul 21 13:13:58.661749 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:58.661722 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq"] Jul 21 13:13:58.729657 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:58.729616 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1544ddb-e5b0-438e-bc98-8313a5313db4-openshift-service-ca\") pod \"maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq\" (UID: \"a1544ddb-e5b0-438e-bc98-8313a5313db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" Jul 21 13:13:58.729885 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:58.729750 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1544ddb-e5b0-438e-bc98-8313a5313db4-maas-api-tls\") pod \"maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq\" (UID: \"a1544ddb-e5b0-438e-bc98-8313a5313db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" Jul 21 13:13:58.730145 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:58.730119 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p4j8\" (UniqueName: \"kubernetes.io/projected/a1544ddb-e5b0-438e-bc98-8313a5313db4-kube-api-access-7p4j8\") pod \"maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq\" (UID: \"a1544ddb-e5b0-438e-bc98-8313a5313db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" Jul 21 13:13:58.831231 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:58.831191 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1544ddb-e5b0-438e-bc98-8313a5313db4-maas-api-tls\") pod \"maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq\" (UID: \"a1544ddb-e5b0-438e-bc98-8313a5313db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" Jul 21 13:13:58.831231 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:58.831240 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p4j8\" (UniqueName: \"kubernetes.io/projected/a1544ddb-e5b0-438e-bc98-8313a5313db4-kube-api-access-7p4j8\") pod \"maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq\" (UID: \"a1544ddb-e5b0-438e-bc98-8313a5313db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" Jul 21 13:13:58.831492 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:58.831315 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1544ddb-e5b0-438e-bc98-8313a5313db4-openshift-service-ca\") pod \"maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq\" (UID: \"a1544ddb-e5b0-438e-bc98-8313a5313db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" Jul 21 13:13:58.831492 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:13:58.831423 2555 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-serving-cert: secret "maas-api-e2e-mt-4f0dda7a-serving-cert" not found Jul 21 13:13:58.831566 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:13:58.831504 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1544ddb-e5b0-438e-bc98-8313a5313db4-maas-api-tls podName:a1544ddb-e5b0-438e-bc98-8313a5313db4 nodeName:}" failed. No retries permitted until 2026-07-21 13:13:59.331484189 +0000 UTC m=+2061.949777115 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a1544ddb-e5b0-438e-bc98-8313a5313db4-maas-api-tls") pod "maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" (UID: "a1544ddb-e5b0-438e-bc98-8313a5313db4") : secret "maas-api-e2e-mt-4f0dda7a-serving-cert" not found Jul 21 13:13:58.832061 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:58.832044 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1544ddb-e5b0-438e-bc98-8313a5313db4-openshift-service-ca\") pod \"maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq\" (UID: \"a1544ddb-e5b0-438e-bc98-8313a5313db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" Jul 21 13:13:58.844197 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:58.844164 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p4j8\" (UniqueName: \"kubernetes.io/projected/a1544ddb-e5b0-438e-bc98-8313a5313db4-kube-api-access-7p4j8\") pod \"maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq\" (UID: \"a1544ddb-e5b0-438e-bc98-8313a5313db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" Jul 21 13:13:59.335072 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:59.335031 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1544ddb-e5b0-438e-bc98-8313a5313db4-maas-api-tls\") pod \"maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq\" (UID: \"a1544ddb-e5b0-438e-bc98-8313a5313db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" Jul 21 13:13:59.337760 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:59.337723 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1544ddb-e5b0-438e-bc98-8313a5313db4-maas-api-tls\") pod \"maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq\" (UID: \"a1544ddb-e5b0-438e-bc98-8313a5313db4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" Jul 21 13:13:59.560775 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:59.560733 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" Jul 21 13:13:59.711465 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:59.711434 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq"] Jul 21 13:13:59.713512 ip-10-0-133-39 kubenswrapper[2555]: W0721 13:13:59.713486 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1544ddb_e5b0_438e_bc98_8313a5313db4.slice/crio-cc465d3afb255c6d43fa8d108f9b6533d69d0111f447fb7f2a3275c51f04785c WatchSource:0}: Error finding container cc465d3afb255c6d43fa8d108f9b6533d69d0111f447fb7f2a3275c51f04785c: Status 404 returned error can't find the container with id cc465d3afb255c6d43fa8d108f9b6533d69d0111f447fb7f2a3275c51f04785c Jul 21 13:13:59.715459 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:13:59.715436 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 13:14:00.596873 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:00.596840 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" event={"ID":"a1544ddb-e5b0-438e-bc98-8313a5313db4","Type":"ContainerStarted","Data":"bf52002c0397b825f7a6f46b911c6c4293bacd8f4b5cfb88632ababb4a257888"} Jul 21 13:14:00.597041 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:00.596897 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" event={"ID":"a1544ddb-e5b0-438e-bc98-8313a5313db4","Type":"ContainerStarted","Data":"cc465d3afb255c6d43fa8d108f9b6533d69d0111f447fb7f2a3275c51f04785c"} Jul 21 13:14:00.597041 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:00.596926 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" Jul 21 13:14:00.618694 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:00.618638 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" podStartSLOduration=2.227755336 podStartE2EDuration="2.618619741s" totalImagesPullingTime="390.864405ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:13:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:14:00.617425903 +0000 UTC m=+2063.235718851" watchObservedRunningTime="2026-07-21 13:14:00.618619741 +0000 UTC m=+2063.236912679" Jul 21 13:14:06.608305 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:06.608274 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" Jul 21 13:14:11.672361 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:11.672266 2555 generic.go:356] "Generic (PLEG): container finished" podID="5ac73392-1a91-4fea-975f-2d145ab20a7c" containerID="d1d5a83119478f2f530aa6b776c2731ca5f23b1178ef1caa38f517ed098d8842" exitCode=0 Jul 21 13:14:11.672819 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:11.672349 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" event={"ID":"5ac73392-1a91-4fea-975f-2d145ab20a7c","Type":"ContainerDied","Data":"d1d5a83119478f2f530aa6b776c2731ca5f23b1178ef1caa38f517ed098d8842"} Jul 21 13:14:11.672819 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:11.672767 2555 scope.go:117] "RemoveContainer" containerID="d1d5a83119478f2f530aa6b776c2731ca5f23b1178ef1caa38f517ed098d8842" Jul 21 13:14:11.673802 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:11.673770 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:14:11.673876 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:11.673856 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:14:11.673921 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:11.673905 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:14:12.579939 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:12.579903 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:14:12.679889 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:12.679857 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" event={"ID":"5ac73392-1a91-4fea-975f-2d145ab20a7c","Type":"ContainerStarted","Data":"76a49dcb2e3aa68b713077b907b9fe7c7b7cd10389367754b9d8baa4e516f8dc"} Jul 21 13:14:13.043629 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:13.043571 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:14:13.045044 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:13.045016 2555 patch_prober.go:28] interesting pod/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 21 13:14:13.045187 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:13.045064 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" podUID="5ac73392-1a91-4fea-975f-2d145ab20a7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 21 13:14:14.043217 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:14.043177 2555 patch_prober.go:28] interesting pod/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 21 13:14:14.043681 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:14.043243 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" podUID="5ac73392-1a91-4fea-975f-2d145ab20a7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 21 13:14:15.043740 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:15.043704 2555 patch_prober.go:28] interesting pod/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 21 13:14:15.044163 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:15.043763 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" podUID="5ac73392-1a91-4fea-975f-2d145ab20a7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 21 13:14:16.043783 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:16.043741 2555 patch_prober.go:28] interesting pod/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 21 13:14:16.044215 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:16.043801 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" podUID="5ac73392-1a91-4fea-975f-2d145ab20a7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 21 13:14:17.047842 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:17.047811 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:14:17.048340 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:17.048194 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:14:17.048930 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:17.048911 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:14:38.003975 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:38.003940 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 13:14:38.008249 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:38.008223 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 13:14:38.009626 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:38.009589 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 13:14:38.014114 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:38.014093 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 13:14:57.289510 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.289476 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq"] Jul 21 13:14:57.290029 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.289702 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" podUID="a1544ddb-e5b0-438e-bc98-8313a5313db4" containerName="maas-api" containerID="cri-o://bf52002c0397b825f7a6f46b911c6c4293bacd8f4b5cfb88632ababb4a257888" gracePeriod=30 Jul 21 13:14:57.542320 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.542254 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" Jul 21 13:14:57.664551 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.664502 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1544ddb-e5b0-438e-bc98-8313a5313db4-openshift-service-ca\") pod \"a1544ddb-e5b0-438e-bc98-8313a5313db4\" (UID: \"a1544ddb-e5b0-438e-bc98-8313a5313db4\") " Jul 21 13:14:57.664779 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.664561 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1544ddb-e5b0-438e-bc98-8313a5313db4-maas-api-tls\") pod \"a1544ddb-e5b0-438e-bc98-8313a5313db4\" (UID: \"a1544ddb-e5b0-438e-bc98-8313a5313db4\") " Jul 21 13:14:57.664779 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.664585 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7p4j8\" (UniqueName: \"kubernetes.io/projected/a1544ddb-e5b0-438e-bc98-8313a5313db4-kube-api-access-7p4j8\") pod \"a1544ddb-e5b0-438e-bc98-8313a5313db4\" (UID: \"a1544ddb-e5b0-438e-bc98-8313a5313db4\") " Jul 21 13:14:57.664985 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.664955 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1544ddb-e5b0-438e-bc98-8313a5313db4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a1544ddb-e5b0-438e-bc98-8313a5313db4" (UID: "a1544ddb-e5b0-438e-bc98-8313a5313db4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:14:57.666941 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.666912 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1544ddb-e5b0-438e-bc98-8313a5313db4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a1544ddb-e5b0-438e-bc98-8313a5313db4" (UID: "a1544ddb-e5b0-438e-bc98-8313a5313db4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:14:57.667065 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.667003 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1544ddb-e5b0-438e-bc98-8313a5313db4-kube-api-access-7p4j8" (OuterVolumeSpecName: "kube-api-access-7p4j8") pod "a1544ddb-e5b0-438e-bc98-8313a5313db4" (UID: "a1544ddb-e5b0-438e-bc98-8313a5313db4"). InnerVolumeSpecName "kube-api-access-7p4j8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:14:57.765420 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.765388 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1544ddb-e5b0-438e-bc98-8313a5313db4-maas-api-tls\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:14:57.765420 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.765414 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7p4j8\" (UniqueName: \"kubernetes.io/projected/a1544ddb-e5b0-438e-bc98-8313a5313db4-kube-api-access-7p4j8\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:14:57.765420 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.765425 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1544ddb-e5b0-438e-bc98-8313a5313db4-openshift-service-ca\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:14:57.919925 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.919886 2555 generic.go:356] "Generic (PLEG): container finished" podID="a1544ddb-e5b0-438e-bc98-8313a5313db4" containerID="bf52002c0397b825f7a6f46b911c6c4293bacd8f4b5cfb88632ababb4a257888" exitCode=0 Jul 21 13:14:57.922391 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.922360 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" Jul 21 13:14:57.929367 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.929321 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" event={"ID":"a1544ddb-e5b0-438e-bc98-8313a5313db4","Type":"ContainerDied","Data":"bf52002c0397b825f7a6f46b911c6c4293bacd8f4b5cfb88632ababb4a257888"} Jul 21 13:14:57.929562 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.929407 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq" event={"ID":"a1544ddb-e5b0-438e-bc98-8313a5313db4","Type":"ContainerDied","Data":"cc465d3afb255c6d43fa8d108f9b6533d69d0111f447fb7f2a3275c51f04785c"} Jul 21 13:14:57.929562 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.929449 2555 scope.go:117] "RemoveContainer" containerID="bf52002c0397b825f7a6f46b911c6c4293bacd8f4b5cfb88632ababb4a257888" Jul 21 13:14:57.941836 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.941814 2555 scope.go:117] "RemoveContainer" containerID="bf52002c0397b825f7a6f46b911c6c4293bacd8f4b5cfb88632ababb4a257888" Jul 21 13:14:57.942151 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:14:57.942121 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf52002c0397b825f7a6f46b911c6c4293bacd8f4b5cfb88632ababb4a257888\": container with ID starting with bf52002c0397b825f7a6f46b911c6c4293bacd8f4b5cfb88632ababb4a257888 not found: ID does not exist" containerID="bf52002c0397b825f7a6f46b911c6c4293bacd8f4b5cfb88632ababb4a257888" Jul 21 13:14:57.942227 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.942158 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf52002c0397b825f7a6f46b911c6c4293bacd8f4b5cfb88632ababb4a257888"} err="failed to get container status \"bf52002c0397b825f7a6f46b911c6c4293bacd8f4b5cfb88632ababb4a257888\": rpc error: code = NotFound desc = could not find container \"bf52002c0397b825f7a6f46b911c6c4293bacd8f4b5cfb88632ababb4a257888\": container with ID starting with bf52002c0397b825f7a6f46b911c6c4293bacd8f4b5cfb88632ababb4a257888 not found: ID does not exist" Jul 21 13:14:57.956359 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.956326 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq"] Jul 21 13:14:57.958456 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:57.958431 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f0dda7a-6895d87b6d-4lcsq"] Jul 21 13:14:59.926944 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:14:59.926906 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1544ddb-e5b0-438e-bc98-8313a5313db4" path="/var/lib/kubelet/pods/a1544ddb-e5b0-438e-bc98-8313a5313db4/volumes" Jul 21 13:15:53.731539 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:53.731493 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph"] Jul 21 13:15:53.732127 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:53.731717 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" podUID="5ac73392-1a91-4fea-975f-2d145ab20a7c" containerName="istio-proxy" containerID="cri-o://76a49dcb2e3aa68b713077b907b9fe7c7b7cd10389367754b9d8baa4e516f8dc" gracePeriod=30 Jul 21 13:15:58.992107 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:58.992076 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:15:59.007889 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.007812 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-podinfo\") pod \"5ac73392-1a91-4fea-975f-2d145ab20a7c\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " Jul 21 13:15:59.007889 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.007858 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-workload-certs\") pod \"5ac73392-1a91-4fea-975f-2d145ab20a7c\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " Jul 21 13:15:59.007889 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.007889 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-workload-socket\") pod \"5ac73392-1a91-4fea-975f-2d145ab20a7c\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " Jul 21 13:15:59.008099 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.007931 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-envoy\") pod \"5ac73392-1a91-4fea-975f-2d145ab20a7c\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " Jul 21 13:15:59.008099 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.007986 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlncc\" (UniqueName: \"kubernetes.io/projected/5ac73392-1a91-4fea-975f-2d145ab20a7c-kube-api-access-rlncc\") pod \"5ac73392-1a91-4fea-975f-2d145ab20a7c\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " Jul 21 13:15:59.008099 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.008022 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ac73392-1a91-4fea-975f-2d145ab20a7c-istiod-ca-cert\") pod \"5ac73392-1a91-4fea-975f-2d145ab20a7c\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " Jul 21 13:15:59.008099 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.008054 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-data\") pod \"5ac73392-1a91-4fea-975f-2d145ab20a7c\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " Jul 21 13:15:59.008099 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.008084 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-credential-socket\") pod \"5ac73392-1a91-4fea-975f-2d145ab20a7c\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " Jul 21 13:15:59.008349 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.008107 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-token\") pod \"5ac73392-1a91-4fea-975f-2d145ab20a7c\" (UID: \"5ac73392-1a91-4fea-975f-2d145ab20a7c\") " Jul 21 13:15:59.008349 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.008214 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5ac73392-1a91-4fea-975f-2d145ab20a7c" (UID: "5ac73392-1a91-4fea-975f-2d145ab20a7c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:15:59.008460 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.008395 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-workload-certs\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:15:59.008460 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.008395 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5ac73392-1a91-4fea-975f-2d145ab20a7c" (UID: "5ac73392-1a91-4fea-975f-2d145ab20a7c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:15:59.008546 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.008499 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ac73392-1a91-4fea-975f-2d145ab20a7c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5ac73392-1a91-4fea-975f-2d145ab20a7c" (UID: "5ac73392-1a91-4fea-975f-2d145ab20a7c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:15:59.008620 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.008578 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5ac73392-1a91-4fea-975f-2d145ab20a7c" (UID: "5ac73392-1a91-4fea-975f-2d145ab20a7c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:15:59.009155 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.009123 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-data" (OuterVolumeSpecName: "istio-data") pod "5ac73392-1a91-4fea-975f-2d145ab20a7c" (UID: "5ac73392-1a91-4fea-975f-2d145ab20a7c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:15:59.010340 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.010311 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5ac73392-1a91-4fea-975f-2d145ab20a7c" (UID: "5ac73392-1a91-4fea-975f-2d145ab20a7c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:15:59.010460 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.010353 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5ac73392-1a91-4fea-975f-2d145ab20a7c" (UID: "5ac73392-1a91-4fea-975f-2d145ab20a7c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 13:15:59.010460 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.010366 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ac73392-1a91-4fea-975f-2d145ab20a7c-kube-api-access-rlncc" (OuterVolumeSpecName: "kube-api-access-rlncc") pod "5ac73392-1a91-4fea-975f-2d145ab20a7c" (UID: "5ac73392-1a91-4fea-975f-2d145ab20a7c"). InnerVolumeSpecName "kube-api-access-rlncc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:15:59.010545 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.010523 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-token" (OuterVolumeSpecName: "istio-token") pod "5ac73392-1a91-4fea-975f-2d145ab20a7c" (UID: "5ac73392-1a91-4fea-975f-2d145ab20a7c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:15:59.109534 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.109491 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ac73392-1a91-4fea-975f-2d145ab20a7c-istiod-ca-cert\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:15:59.109534 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.109523 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-data\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:15:59.109534 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.109532 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-credential-socket\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:15:59.109534 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.109541 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-token\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:15:59.109849 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.109549 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-podinfo\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:15:59.109849 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.109557 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-workload-socket\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:15:59.109849 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.109565 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ac73392-1a91-4fea-975f-2d145ab20a7c-istio-envoy\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:15:59.109849 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.109575 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlncc\" (UniqueName: \"kubernetes.io/projected/5ac73392-1a91-4fea-975f-2d145ab20a7c-kube-api-access-rlncc\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:15:59.262285 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.262192 2555 generic.go:356] "Generic (PLEG): container finished" podID="5ac73392-1a91-4fea-975f-2d145ab20a7c" containerID="76a49dcb2e3aa68b713077b907b9fe7c7b7cd10389367754b9d8baa4e516f8dc" exitCode=0 Jul 21 13:15:59.262285 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.262246 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" Jul 21 13:15:59.262471 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.262276 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" event={"ID":"5ac73392-1a91-4fea-975f-2d145ab20a7c","Type":"ContainerDied","Data":"76a49dcb2e3aa68b713077b907b9fe7c7b7cd10389367754b9d8baa4e516f8dc"} Jul 21 13:15:59.262471 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.262329 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph" event={"ID":"5ac73392-1a91-4fea-975f-2d145ab20a7c","Type":"ContainerDied","Data":"d38f7d47362ca68b42e4acffbf7eac012b5c4999133e39aaeac1a4cfade8e7d6"} Jul 21 13:15:59.262471 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.262353 2555 scope.go:117] "RemoveContainer" containerID="76a49dcb2e3aa68b713077b907b9fe7c7b7cd10389367754b9d8baa4e516f8dc" Jul 21 13:15:59.272446 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.272430 2555 scope.go:117] "RemoveContainer" containerID="d1d5a83119478f2f530aa6b776c2731ca5f23b1178ef1caa38f517ed098d8842" Jul 21 13:15:59.280905 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.280887 2555 scope.go:117] "RemoveContainer" containerID="76a49dcb2e3aa68b713077b907b9fe7c7b7cd10389367754b9d8baa4e516f8dc" Jul 21 13:15:59.281188 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:15:59.281166 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76a49dcb2e3aa68b713077b907b9fe7c7b7cd10389367754b9d8baa4e516f8dc\": container with ID starting with 76a49dcb2e3aa68b713077b907b9fe7c7b7cd10389367754b9d8baa4e516f8dc not found: ID does not exist" containerID="76a49dcb2e3aa68b713077b907b9fe7c7b7cd10389367754b9d8baa4e516f8dc" Jul 21 13:15:59.281287 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.281196 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76a49dcb2e3aa68b713077b907b9fe7c7b7cd10389367754b9d8baa4e516f8dc"} err="failed to get container status \"76a49dcb2e3aa68b713077b907b9fe7c7b7cd10389367754b9d8baa4e516f8dc\": rpc error: code = NotFound desc = could not find container \"76a49dcb2e3aa68b713077b907b9fe7c7b7cd10389367754b9d8baa4e516f8dc\": container with ID starting with 76a49dcb2e3aa68b713077b907b9fe7c7b7cd10389367754b9d8baa4e516f8dc not found: ID does not exist" Jul 21 13:15:59.281287 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.281215 2555 scope.go:117] "RemoveContainer" containerID="d1d5a83119478f2f530aa6b776c2731ca5f23b1178ef1caa38f517ed098d8842" Jul 21 13:15:59.281458 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:15:59.281442 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1d5a83119478f2f530aa6b776c2731ca5f23b1178ef1caa38f517ed098d8842\": container with ID starting with d1d5a83119478f2f530aa6b776c2731ca5f23b1178ef1caa38f517ed098d8842 not found: ID does not exist" containerID="d1d5a83119478f2f530aa6b776c2731ca5f23b1178ef1caa38f517ed098d8842" Jul 21 13:15:59.281498 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.281465 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1d5a83119478f2f530aa6b776c2731ca5f23b1178ef1caa38f517ed098d8842"} err="failed to get container status \"d1d5a83119478f2f530aa6b776c2731ca5f23b1178ef1caa38f517ed098d8842\": rpc error: code = NotFound desc = could not find container \"d1d5a83119478f2f530aa6b776c2731ca5f23b1178ef1caa38f517ed098d8842\": container with ID starting with d1d5a83119478f2f530aa6b776c2731ca5f23b1178ef1caa38f517ed098d8842 not found: ID does not exist" Jul 21 13:15:59.289782 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.289758 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph"] Jul 21 13:15:59.298926 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.298904 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4f0dda7a-openshift-default-74fcf7894c-6ktph"] Jul 21 13:15:59.927527 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:15:59.927490 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ac73392-1a91-4fea-975f-2d145ab20a7c" path="/var/lib/kubelet/pods/5ac73392-1a91-4fea-975f-2d145ab20a7c/volumes" Jul 21 13:16:52.057800 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.057762 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4"] Jul 21 13:16:52.058269 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.058178 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1544ddb-e5b0-438e-bc98-8313a5313db4" containerName="maas-api" Jul 21 13:16:52.058269 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.058191 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1544ddb-e5b0-438e-bc98-8313a5313db4" containerName="maas-api" Jul 21 13:16:52.058269 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.058205 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ac73392-1a91-4fea-975f-2d145ab20a7c" containerName="istio-proxy" Jul 21 13:16:52.058269 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.058211 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ac73392-1a91-4fea-975f-2d145ab20a7c" containerName="istio-proxy" Jul 21 13:16:52.058269 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.058217 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ac73392-1a91-4fea-975f-2d145ab20a7c" containerName="istio-proxy" Jul 21 13:16:52.058269 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.058224 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ac73392-1a91-4fea-975f-2d145ab20a7c" containerName="istio-proxy" Jul 21 13:16:52.058487 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.058284 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ac73392-1a91-4fea-975f-2d145ab20a7c" containerName="istio-proxy" Jul 21 13:16:52.058487 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.058295 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1544ddb-e5b0-438e-bc98-8313a5313db4" containerName="maas-api" Jul 21 13:16:52.061646 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.061620 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.064161 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.064142 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-360ca4-openshift-default-dockercfg-tm88c\"" Jul 21 13:16:52.067533 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.067507 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-data\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.067658 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.067566 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1624f09f-5534-4043-8472-1f09ab94b6c2-istiod-ca-cert\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.067658 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.067618 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-envoy\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.067658 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.067651 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-workload-certs\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.067793 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.067713 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-credential-socket\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.067793 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.067754 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-workload-socket\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.067793 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.067781 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-podinfo\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.067950 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.067829 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqw7j\" (UniqueName: \"kubernetes.io/projected/1624f09f-5534-4043-8472-1f09ab94b6c2-kube-api-access-vqw7j\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.067950 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.067852 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-token\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.079920 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.079885 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4"] Jul 21 13:16:52.168219 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.168184 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-workload-socket\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.168219 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.168223 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-podinfo\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.168486 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.168248 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqw7j\" (UniqueName: \"kubernetes.io/projected/1624f09f-5534-4043-8472-1f09ab94b6c2-kube-api-access-vqw7j\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.168486 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.168268 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-token\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.168486 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.168298 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-data\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.168486 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.168341 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1624f09f-5534-4043-8472-1f09ab94b6c2-istiod-ca-cert\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.168486 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.168372 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-envoy\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.168486 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.168401 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-workload-certs\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.168847 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.168487 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-credential-socket\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.168847 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.168746 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-workload-socket\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.168847 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.168801 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-data\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.169028 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.168998 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-credential-socket\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.169028 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.169018 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-workload-certs\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.169191 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.169108 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1624f09f-5534-4043-8472-1f09ab94b6c2-istiod-ca-cert\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.171581 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.171556 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-envoy\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.171711 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.171696 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-podinfo\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.177686 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.177663 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-token\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.177790 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.177726 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqw7j\" (UniqueName: \"kubernetes.io/projected/1624f09f-5534-4043-8472-1f09ab94b6c2-kube-api-access-vqw7j\") pod \"e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.373339 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.373295 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:52.524420 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.524391 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4"] Jul 21 13:16:52.526861 ip-10-0-133-39 kubenswrapper[2555]: W0721 13:16:52.526832 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1624f09f_5534_4043_8472_1f09ab94b6c2.slice/crio-25afa2955e35b9f9f67d8a54191b799db0f95407b2aa14d9223fdc9c71e2249d WatchSource:0}: Error finding container 25afa2955e35b9f9f67d8a54191b799db0f95407b2aa14d9223fdc9c71e2249d: Status 404 returned error can't find the container with id 25afa2955e35b9f9f67d8a54191b799db0f95407b2aa14d9223fdc9c71e2249d Jul 21 13:16:52.529533 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.529490 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:16:52.529658 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.529634 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:16:52.529710 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.529680 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:16:52.543181 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:52.543148 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" event={"ID":"1624f09f-5534-4043-8472-1f09ab94b6c2","Type":"ContainerStarted","Data":"25afa2955e35b9f9f67d8a54191b799db0f95407b2aa14d9223fdc9c71e2249d"} Jul 21 13:16:53.550524 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:53.550491 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" event={"ID":"1624f09f-5534-4043-8472-1f09ab94b6c2","Type":"ContainerStarted","Data":"9dcc2a409cffd86711d6b73b7a0374ff8c7f78cdf3fd6ba3eb6dc1fa259d4205"} Jul 21 13:16:53.579323 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:53.579263 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" podStartSLOduration=1.579236519 podStartE2EDuration="1.579236519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:16:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:16:53.57714361 +0000 UTC m=+2236.195436570" watchObservedRunningTime="2026-07-21 13:16:53.579236519 +0000 UTC m=+2236.197529468" Jul 21 13:16:54.373896 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:54.373853 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:54.379025 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:54.378994 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:54.556122 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:54.556088 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:16:54.557037 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:16:54.557015 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:17:00.862238 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:00.862155 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq"] Jul 21 13:17:00.862653 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:00.862618 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ac73392-1a91-4fea-975f-2d145ab20a7c" containerName="istio-proxy" Jul 21 13:17:00.865993 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:00.865976 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:00.868757 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:00.868731 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-823a18-openshift-default-dockercfg-hkjz5\"" Jul 21 13:17:00.885432 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:00.885399 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq"] Jul 21 13:17:00.941683 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:00.941636 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-workload-certs\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:00.941683 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:00.941695 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-credential-socket\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:00.941928 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:00.941716 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-envoy\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:00.941928 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:00.941756 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-data\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:00.941928 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:00.941813 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-token\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:00.941928 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:00.941827 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22f64ca3-f820-4609-aba3-22c9ea79636a-istiod-ca-cert\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:00.941928 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:00.941845 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvhdk\" (UniqueName: \"kubernetes.io/projected/22f64ca3-f820-4609-aba3-22c9ea79636a-kube-api-access-kvhdk\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:00.941928 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:00.941862 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-workload-socket\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:00.942204 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:00.941953 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-podinfo\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.042662 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.042584 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-podinfo\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.042890 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.042673 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-workload-certs\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.042890 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.042718 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-credential-socket\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.042890 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.042745 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-envoy\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.042890 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.042829 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-data\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.043126 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.042940 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-token\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.043126 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.042966 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22f64ca3-f820-4609-aba3-22c9ea79636a-istiod-ca-cert\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.043126 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.042992 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvhdk\" (UniqueName: \"kubernetes.io/projected/22f64ca3-f820-4609-aba3-22c9ea79636a-kube-api-access-kvhdk\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.043126 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.043017 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-workload-socket\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.043354 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.043281 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-credential-socket\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.043354 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.043313 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-data\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.043715 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.043649 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-workload-socket\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.043715 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.043679 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-workload-certs\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.044067 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.044044 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22f64ca3-f820-4609-aba3-22c9ea79636a-istiod-ca-cert\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.047069 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.047033 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-envoy\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.047654 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.047628 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-podinfo\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.057266 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.057230 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-token\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.057451 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.057381 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvhdk\" (UniqueName: \"kubernetes.io/projected/22f64ca3-f820-4609-aba3-22c9ea79636a-kube-api-access-kvhdk\") pod \"e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.177225 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.177117 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:01.344816 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.344780 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq"] Jul 21 13:17:01.346243 ip-10-0-133-39 kubenswrapper[2555]: W0721 13:17:01.346210 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22f64ca3_f820_4609_aba3_22c9ea79636a.slice/crio-1e06d83df0d3cf5ba1ed8119864b4241cdaa193c435a290b0c0ddb558942e8e8 WatchSource:0}: Error finding container 1e06d83df0d3cf5ba1ed8119864b4241cdaa193c435a290b0c0ddb558942e8e8: Status 404 returned error can't find the container with id 1e06d83df0d3cf5ba1ed8119864b4241cdaa193c435a290b0c0ddb558942e8e8 Jul 21 13:17:01.348403 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.348366 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:17:01.348489 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.348439 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:17:01.348489 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.348468 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:17:01.601442 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.601397 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" event={"ID":"22f64ca3-f820-4609-aba3-22c9ea79636a","Type":"ContainerStarted","Data":"6a0005aeddf67da427a4c6a25866fd5e271158031a8dfd79f8d68a0366a3f21d"} Jul 21 13:17:01.601708 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.601467 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" event={"ID":"22f64ca3-f820-4609-aba3-22c9ea79636a","Type":"ContainerStarted","Data":"1e06d83df0d3cf5ba1ed8119864b4241cdaa193c435a290b0c0ddb558942e8e8"} Jul 21 13:17:01.643636 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:01.643555 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" podStartSLOduration=1.6435359680000001 podStartE2EDuration="1.643535968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:17:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:17:01.641774682 +0000 UTC m=+2244.260067630" watchObservedRunningTime="2026-07-21 13:17:01.643535968 +0000 UTC m=+2244.261828989" Jul 21 13:17:02.177564 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:02.177530 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:02.182748 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:02.182719 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:02.606625 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:02.606583 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:02.607717 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:02.607698 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:09.911976 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:09.911773 2555 patch_prober.go:28] interesting pod/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": read tcp 10.132.0.2:35782->10.132.0.54:15021: read: connection reset by peer" start-of-body= Jul 21 13:17:09.911976 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:09.911839 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" podUID="1624f09f-5534-4043-8472-1f09ab94b6c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": read tcp 10.132.0.2:35782->10.132.0.54:15021: read: connection reset by peer" Jul 21 13:17:10.672059 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:10.672023 2555 generic.go:356] "Generic (PLEG): container finished" podID="1624f09f-5534-4043-8472-1f09ab94b6c2" containerID="9dcc2a409cffd86711d6b73b7a0374ff8c7f78cdf3fd6ba3eb6dc1fa259d4205" exitCode=0 Jul 21 13:17:10.672258 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:10.672099 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" event={"ID":"1624f09f-5534-4043-8472-1f09ab94b6c2","Type":"ContainerDied","Data":"9dcc2a409cffd86711d6b73b7a0374ff8c7f78cdf3fd6ba3eb6dc1fa259d4205"} Jul 21 13:17:10.672519 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:10.672506 2555 scope.go:117] "RemoveContainer" containerID="9dcc2a409cffd86711d6b73b7a0374ff8c7f78cdf3fd6ba3eb6dc1fa259d4205" Jul 21 13:17:10.673529 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:10.673499 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:17:10.673585 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:10.673569 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:17:10.673647 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:10.673631 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:17:11.679869 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:11.679835 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" event={"ID":"1624f09f-5534-4043-8472-1f09ab94b6c2","Type":"ContainerStarted","Data":"23e05a6d98431076d5b5ed1fb771140d1e6495846ca129b7d9469ac0a5117f5a"} Jul 21 13:17:12.374108 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:12.374070 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:17:12.374326 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:12.374192 2555 patch_prober.go:28] interesting pod/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 21 13:17:12.374326 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:12.374235 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" podUID="1624f09f-5534-4043-8472-1f09ab94b6c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 21 13:17:13.373927 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:13.373890 2555 patch_prober.go:28] interesting pod/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 21 13:17:13.374343 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:13.373947 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" podUID="1624f09f-5534-4043-8472-1f09ab94b6c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 21 13:17:14.374554 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:14.374518 2555 patch_prober.go:28] interesting pod/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 21 13:17:14.375088 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:14.374748 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" podUID="1624f09f-5534-4043-8472-1f09ab94b6c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 21 13:17:15.374315 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:15.374278 2555 patch_prober.go:28] interesting pod/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 21 13:17:15.374503 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:15.374332 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" podUID="1624f09f-5534-4043-8472-1f09ab94b6c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 21 13:17:16.377753 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:16.377722 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:17:16.378156 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:16.378131 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:17:16.378944 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:16.378927 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:17:18.647826 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:18.647783 2555 patch_prober.go:28] interesting pod/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 21 13:17:18.648245 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:18.647851 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" podUID="22f64ca3-f820-4609-aba3-22c9ea79636a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 21 13:17:19.730668 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:19.730638 2555 generic.go:356] "Generic (PLEG): container finished" podID="22f64ca3-f820-4609-aba3-22c9ea79636a" containerID="6a0005aeddf67da427a4c6a25866fd5e271158031a8dfd79f8d68a0366a3f21d" exitCode=0 Jul 21 13:17:19.731139 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:19.730710 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" event={"ID":"22f64ca3-f820-4609-aba3-22c9ea79636a","Type":"ContainerDied","Data":"6a0005aeddf67da427a4c6a25866fd5e271158031a8dfd79f8d68a0366a3f21d"} Jul 21 13:17:19.731139 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:19.731065 2555 scope.go:117] "RemoveContainer" containerID="6a0005aeddf67da427a4c6a25866fd5e271158031a8dfd79f8d68a0366a3f21d" Jul 21 13:17:19.732049 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:19.732023 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:17:19.732105 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:19.732092 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:17:19.732143 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:19.732131 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:17:20.742759 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:20.742719 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" event={"ID":"22f64ca3-f820-4609-aba3-22c9ea79636a","Type":"ContainerStarted","Data":"76da034c4c76005645524299bf940e589a7ffe1de684217d8aa7d14f39f3821b"} Jul 21 13:17:21.178245 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:21.178205 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:21.178453 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:21.178431 2555 patch_prober.go:28] interesting pod/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 21 13:17:21.178501 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:21.178479 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" podUID="22f64ca3-f820-4609-aba3-22c9ea79636a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 21 13:17:22.178496 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:22.178455 2555 patch_prober.go:28] interesting pod/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 21 13:17:22.178916 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:22.178518 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" podUID="22f64ca3-f820-4609-aba3-22c9ea79636a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 21 13:17:23.177801 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:23.177764 2555 patch_prober.go:28] interesting pod/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 21 13:17:23.178055 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:23.177825 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" podUID="22f64ca3-f820-4609-aba3-22c9ea79636a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 21 13:17:24.178554 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:24.178504 2555 patch_prober.go:28] interesting pod/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 21 13:17:24.179011 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:24.178575 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" podUID="22f64ca3-f820-4609-aba3-22c9ea79636a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 21 13:17:25.181029 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:25.180995 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:25.181432 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:25.181315 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:17:25.182300 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:17:25.182279 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:18:41.445234 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:41.445132 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4"] Jul 21 13:18:41.445784 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:41.445358 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" podUID="1624f09f-5534-4043-8472-1f09ab94b6c2" containerName="istio-proxy" containerID="cri-o://23e05a6d98431076d5b5ed1fb771140d1e6495846ca129b7d9469ac0a5117f5a" gracePeriod=30 Jul 21 13:18:46.380791 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.380748 2555 patch_prober.go:28] interesting pod/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 21 13:18:46.381178 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.380822 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" podUID="1624f09f-5534-4043-8472-1f09ab94b6c2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 21 13:18:46.702648 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.702619 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:18:46.808413 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.808370 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-envoy\") pod \"1624f09f-5534-4043-8472-1f09ab94b6c2\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " Jul 21 13:18:46.808413 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.808414 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-token\") pod \"1624f09f-5534-4043-8472-1f09ab94b6c2\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " Jul 21 13:18:46.808704 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.808471 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-data\") pod \"1624f09f-5534-4043-8472-1f09ab94b6c2\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " Jul 21 13:18:46.808704 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.808487 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1624f09f-5534-4043-8472-1f09ab94b6c2-istiod-ca-cert\") pod \"1624f09f-5534-4043-8472-1f09ab94b6c2\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " Jul 21 13:18:46.808704 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.808504 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-workload-socket\") pod \"1624f09f-5534-4043-8472-1f09ab94b6c2\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " Jul 21 13:18:46.808704 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.808531 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-credential-socket\") pod \"1624f09f-5534-4043-8472-1f09ab94b6c2\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " Jul 21 13:18:46.808704 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.808688 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-podinfo\") pod \"1624f09f-5534-4043-8472-1f09ab94b6c2\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " Jul 21 13:18:46.808955 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.808790 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-workload-certs\") pod \"1624f09f-5534-4043-8472-1f09ab94b6c2\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " Jul 21 13:18:46.808955 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.808818 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqw7j\" (UniqueName: \"kubernetes.io/projected/1624f09f-5534-4043-8472-1f09ab94b6c2-kube-api-access-vqw7j\") pod \"1624f09f-5534-4043-8472-1f09ab94b6c2\" (UID: \"1624f09f-5534-4043-8472-1f09ab94b6c2\") " Jul 21 13:18:46.808955 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.808841 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1624f09f-5534-4043-8472-1f09ab94b6c2" (UID: "1624f09f-5534-4043-8472-1f09ab94b6c2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:18:46.809163 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.809043 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1624f09f-5534-4043-8472-1f09ab94b6c2" (UID: "1624f09f-5534-4043-8472-1f09ab94b6c2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:18:46.809163 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.809143 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-workload-certs\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:18:46.809163 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.809160 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-credential-socket\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:18:46.809334 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.809160 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1624f09f-5534-4043-8472-1f09ab94b6c2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1624f09f-5534-4043-8472-1f09ab94b6c2" (UID: "1624f09f-5534-4043-8472-1f09ab94b6c2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:18:46.809334 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.809317 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1624f09f-5534-4043-8472-1f09ab94b6c2" (UID: "1624f09f-5534-4043-8472-1f09ab94b6c2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:18:46.809710 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.809686 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-data" (OuterVolumeSpecName: "istio-data") pod "1624f09f-5534-4043-8472-1f09ab94b6c2" (UID: "1624f09f-5534-4043-8472-1f09ab94b6c2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:18:46.811024 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.810991 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1624f09f-5534-4043-8472-1f09ab94b6c2" (UID: "1624f09f-5534-4043-8472-1f09ab94b6c2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 13:18:46.811218 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.811197 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1624f09f-5534-4043-8472-1f09ab94b6c2-kube-api-access-vqw7j" (OuterVolumeSpecName: "kube-api-access-vqw7j") pod "1624f09f-5534-4043-8472-1f09ab94b6c2" (UID: "1624f09f-5534-4043-8472-1f09ab94b6c2"). InnerVolumeSpecName "kube-api-access-vqw7j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:18:46.811380 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.811362 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1624f09f-5534-4043-8472-1f09ab94b6c2" (UID: "1624f09f-5534-4043-8472-1f09ab94b6c2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:18:46.811428 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.811363 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-token" (OuterVolumeSpecName: "istio-token") pod "1624f09f-5534-4043-8472-1f09ab94b6c2" (UID: "1624f09f-5534-4043-8472-1f09ab94b6c2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:18:46.910022 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.909983 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-data\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:18:46.910022 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.910015 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1624f09f-5534-4043-8472-1f09ab94b6c2-istiod-ca-cert\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:18:46.910022 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.910029 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-workload-socket\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:18:46.910281 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.910039 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-podinfo\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:18:46.910281 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.910052 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqw7j\" (UniqueName: \"kubernetes.io/projected/1624f09f-5534-4043-8472-1f09ab94b6c2-kube-api-access-vqw7j\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:18:46.910281 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.910063 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-envoy\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:18:46.910281 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:46.910074 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1624f09f-5534-4043-8472-1f09ab94b6c2-istio-token\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:18:47.236041 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:47.236001 2555 generic.go:356] "Generic (PLEG): container finished" podID="1624f09f-5534-4043-8472-1f09ab94b6c2" containerID="23e05a6d98431076d5b5ed1fb771140d1e6495846ca129b7d9469ac0a5117f5a" exitCode=0 Jul 21 13:18:47.236253 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:47.236060 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" Jul 21 13:18:47.236253 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:47.236082 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" event={"ID":"1624f09f-5534-4043-8472-1f09ab94b6c2","Type":"ContainerDied","Data":"23e05a6d98431076d5b5ed1fb771140d1e6495846ca129b7d9469ac0a5117f5a"} Jul 21 13:18:47.236253 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:47.236133 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4" event={"ID":"1624f09f-5534-4043-8472-1f09ab94b6c2","Type":"ContainerDied","Data":"25afa2955e35b9f9f67d8a54191b799db0f95407b2aa14d9223fdc9c71e2249d"} Jul 21 13:18:47.236253 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:47.236157 2555 scope.go:117] "RemoveContainer" containerID="23e05a6d98431076d5b5ed1fb771140d1e6495846ca129b7d9469ac0a5117f5a" Jul 21 13:18:47.246234 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:47.246216 2555 scope.go:117] "RemoveContainer" containerID="9dcc2a409cffd86711d6b73b7a0374ff8c7f78cdf3fd6ba3eb6dc1fa259d4205" Jul 21 13:18:47.255080 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:47.255058 2555 scope.go:117] "RemoveContainer" containerID="23e05a6d98431076d5b5ed1fb771140d1e6495846ca129b7d9469ac0a5117f5a" Jul 21 13:18:47.255421 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:18:47.255397 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23e05a6d98431076d5b5ed1fb771140d1e6495846ca129b7d9469ac0a5117f5a\": container with ID starting with 23e05a6d98431076d5b5ed1fb771140d1e6495846ca129b7d9469ac0a5117f5a not found: ID does not exist" containerID="23e05a6d98431076d5b5ed1fb771140d1e6495846ca129b7d9469ac0a5117f5a" Jul 21 13:18:47.255490 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:47.255432 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23e05a6d98431076d5b5ed1fb771140d1e6495846ca129b7d9469ac0a5117f5a"} err="failed to get container status \"23e05a6d98431076d5b5ed1fb771140d1e6495846ca129b7d9469ac0a5117f5a\": rpc error: code = NotFound desc = could not find container \"23e05a6d98431076d5b5ed1fb771140d1e6495846ca129b7d9469ac0a5117f5a\": container with ID starting with 23e05a6d98431076d5b5ed1fb771140d1e6495846ca129b7d9469ac0a5117f5a not found: ID does not exist" Jul 21 13:18:47.255490 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:47.255451 2555 scope.go:117] "RemoveContainer" containerID="9dcc2a409cffd86711d6b73b7a0374ff8c7f78cdf3fd6ba3eb6dc1fa259d4205" Jul 21 13:18:47.255780 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:18:47.255758 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dcc2a409cffd86711d6b73b7a0374ff8c7f78cdf3fd6ba3eb6dc1fa259d4205\": container with ID starting with 9dcc2a409cffd86711d6b73b7a0374ff8c7f78cdf3fd6ba3eb6dc1fa259d4205 not found: ID does not exist" containerID="9dcc2a409cffd86711d6b73b7a0374ff8c7f78cdf3fd6ba3eb6dc1fa259d4205" Jul 21 13:18:47.255841 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:47.255785 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dcc2a409cffd86711d6b73b7a0374ff8c7f78cdf3fd6ba3eb6dc1fa259d4205"} err="failed to get container status \"9dcc2a409cffd86711d6b73b7a0374ff8c7f78cdf3fd6ba3eb6dc1fa259d4205\": rpc error: code = NotFound desc = could not find container \"9dcc2a409cffd86711d6b73b7a0374ff8c7f78cdf3fd6ba3eb6dc1fa259d4205\": container with ID starting with 9dcc2a409cffd86711d6b73b7a0374ff8c7f78cdf3fd6ba3eb6dc1fa259d4205 not found: ID does not exist" Jul 21 13:18:47.261517 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:47.261486 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4"] Jul 21 13:18:47.266919 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:47.266890 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-360ca4-openshift-default-5c4d779555-wsdr4"] Jul 21 13:18:47.926992 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:18:47.926957 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1624f09f-5534-4043-8472-1f09ab94b6c2" path="/var/lib/kubelet/pods/1624f09f-5534-4043-8472-1f09ab94b6c2/volumes" Jul 21 13:19:04.220816 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:04.220781 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq"] Jul 21 13:19:04.221257 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:04.221067 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" podUID="22f64ca3-f820-4609-aba3-22c9ea79636a" containerName="istio-proxy" containerID="cri-o://76da034c4c76005645524299bf940e589a7ffe1de684217d8aa7d14f39f3821b" gracePeriod=30 Jul 21 13:19:08.761049 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:08.761014 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf"] Jul 21 13:19:08.761536 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:08.761454 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1624f09f-5534-4043-8472-1f09ab94b6c2" containerName="istio-proxy" Jul 21 13:19:08.761536 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:08.761466 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="1624f09f-5534-4043-8472-1f09ab94b6c2" containerName="istio-proxy" Jul 21 13:19:08.761679 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:08.761550 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="1624f09f-5534-4043-8472-1f09ab94b6c2" containerName="istio-proxy" Jul 21 13:19:08.761679 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:08.761565 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="1624f09f-5534-4043-8472-1f09ab94b6c2" containerName="istio-proxy" Jul 21 13:19:08.766544 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:08.766521 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" Jul 21 13:19:08.768850 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:08.768826 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-d921bc-serving-cert\"" Jul 21 13:19:08.768850 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:08.768844 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 13:19:08.769130 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:08.769115 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 13:19:08.769584 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:08.769566 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-njpqk\"" Jul 21 13:19:08.776066 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:08.776040 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf"] Jul 21 13:19:08.912559 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:08.912521 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceefb3e4-347d-4d93-8f31-f9212e0fa469-openshift-service-ca\") pod \"maas-api-e2e-api-a-d921bc-866857c7cf-vklxf\" (UID: \"ceefb3e4-347d-4d93-8f31-f9212e0fa469\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" Jul 21 13:19:08.912786 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:08.912585 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45sfq\" (UniqueName: \"kubernetes.io/projected/ceefb3e4-347d-4d93-8f31-f9212e0fa469-kube-api-access-45sfq\") pod \"maas-api-e2e-api-a-d921bc-866857c7cf-vklxf\" (UID: \"ceefb3e4-347d-4d93-8f31-f9212e0fa469\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" Jul 21 13:19:08.912786 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:08.912660 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceefb3e4-347d-4d93-8f31-f9212e0fa469-maas-api-tls\") pod \"maas-api-e2e-api-a-d921bc-866857c7cf-vklxf\" (UID: \"ceefb3e4-347d-4d93-8f31-f9212e0fa469\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" Jul 21 13:19:09.013981 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.013880 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceefb3e4-347d-4d93-8f31-f9212e0fa469-openshift-service-ca\") pod \"maas-api-e2e-api-a-d921bc-866857c7cf-vklxf\" (UID: \"ceefb3e4-347d-4d93-8f31-f9212e0fa469\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" Jul 21 13:19:09.013981 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.013952 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45sfq\" (UniqueName: \"kubernetes.io/projected/ceefb3e4-347d-4d93-8f31-f9212e0fa469-kube-api-access-45sfq\") pod \"maas-api-e2e-api-a-d921bc-866857c7cf-vklxf\" (UID: \"ceefb3e4-347d-4d93-8f31-f9212e0fa469\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" Jul 21 13:19:09.014208 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.013994 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceefb3e4-347d-4d93-8f31-f9212e0fa469-maas-api-tls\") pod \"maas-api-e2e-api-a-d921bc-866857c7cf-vklxf\" (UID: \"ceefb3e4-347d-4d93-8f31-f9212e0fa469\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" Jul 21 13:19:09.014208 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:19:09.014128 2555 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-serving-cert: secret "maas-api-e2e-api-a-d921bc-serving-cert" not found Jul 21 13:19:09.014208 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:19:09.014203 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ceefb3e4-347d-4d93-8f31-f9212e0fa469-maas-api-tls podName:ceefb3e4-347d-4d93-8f31-f9212e0fa469 nodeName:}" failed. No retries permitted until 2026-07-21 13:19:09.514181805 +0000 UTC m=+2372.132474745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ceefb3e4-347d-4d93-8f31-f9212e0fa469-maas-api-tls") pod "maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" (UID: "ceefb3e4-347d-4d93-8f31-f9212e0fa469") : secret "maas-api-e2e-api-a-d921bc-serving-cert" not found Jul 21 13:19:09.014536 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.014515 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceefb3e4-347d-4d93-8f31-f9212e0fa469-openshift-service-ca\") pod \"maas-api-e2e-api-a-d921bc-866857c7cf-vklxf\" (UID: \"ceefb3e4-347d-4d93-8f31-f9212e0fa469\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" Jul 21 13:19:09.028227 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.028198 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45sfq\" (UniqueName: \"kubernetes.io/projected/ceefb3e4-347d-4d93-8f31-f9212e0fa469-kube-api-access-45sfq\") pod \"maas-api-e2e-api-a-d921bc-866857c7cf-vklxf\" (UID: \"ceefb3e4-347d-4d93-8f31-f9212e0fa469\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" Jul 21 13:19:09.361744 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.361713 2555 generic.go:356] "Generic (PLEG): container finished" podID="22f64ca3-f820-4609-aba3-22c9ea79636a" containerID="76da034c4c76005645524299bf940e589a7ffe1de684217d8aa7d14f39f3821b" exitCode=0 Jul 21 13:19:09.361888 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.361785 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" event={"ID":"22f64ca3-f820-4609-aba3-22c9ea79636a","Type":"ContainerDied","Data":"76da034c4c76005645524299bf940e589a7ffe1de684217d8aa7d14f39f3821b"} Jul 21 13:19:09.361946 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.361905 2555 scope.go:117] "RemoveContainer" containerID="6a0005aeddf67da427a4c6a25866fd5e271158031a8dfd79f8d68a0366a3f21d" Jul 21 13:19:09.493375 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.493348 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:19:09.519010 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.518969 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceefb3e4-347d-4d93-8f31-f9212e0fa469-maas-api-tls\") pod \"maas-api-e2e-api-a-d921bc-866857c7cf-vklxf\" (UID: \"ceefb3e4-347d-4d93-8f31-f9212e0fa469\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" Jul 21 13:19:09.523364 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.523294 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceefb3e4-347d-4d93-8f31-f9212e0fa469-maas-api-tls\") pod \"maas-api-e2e-api-a-d921bc-866857c7cf-vklxf\" (UID: \"ceefb3e4-347d-4d93-8f31-f9212e0fa469\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" Jul 21 13:19:09.619890 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.619799 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-workload-socket\") pod \"22f64ca3-f820-4609-aba3-22c9ea79636a\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " Jul 21 13:19:09.619890 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.619839 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22f64ca3-f820-4609-aba3-22c9ea79636a-istiod-ca-cert\") pod \"22f64ca3-f820-4609-aba3-22c9ea79636a\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " Jul 21 13:19:09.619890 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.619856 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-podinfo\") pod \"22f64ca3-f820-4609-aba3-22c9ea79636a\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " Jul 21 13:19:09.619890 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.619880 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-token\") pod \"22f64ca3-f820-4609-aba3-22c9ea79636a\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " Jul 21 13:19:09.620232 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.619943 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-credential-socket\") pod \"22f64ca3-f820-4609-aba3-22c9ea79636a\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " Jul 21 13:19:09.620232 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.619960 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-envoy\") pod \"22f64ca3-f820-4609-aba3-22c9ea79636a\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " Jul 21 13:19:09.620232 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.619979 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvhdk\" (UniqueName: \"kubernetes.io/projected/22f64ca3-f820-4609-aba3-22c9ea79636a-kube-api-access-kvhdk\") pod \"22f64ca3-f820-4609-aba3-22c9ea79636a\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " Jul 21 13:19:09.620232 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.620031 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-workload-certs\") pod \"22f64ca3-f820-4609-aba3-22c9ea79636a\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " Jul 21 13:19:09.620232 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.620051 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-data\") pod \"22f64ca3-f820-4609-aba3-22c9ea79636a\" (UID: \"22f64ca3-f820-4609-aba3-22c9ea79636a\") " Jul 21 13:19:09.620232 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.620194 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "22f64ca3-f820-4609-aba3-22c9ea79636a" (UID: "22f64ca3-f820-4609-aba3-22c9ea79636a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:19:09.620645 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.620336 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-workload-socket\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:19:09.620645 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.620436 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22f64ca3-f820-4609-aba3-22c9ea79636a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "22f64ca3-f820-4609-aba3-22c9ea79636a" (UID: "22f64ca3-f820-4609-aba3-22c9ea79636a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:19:09.620779 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.620707 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "22f64ca3-f820-4609-aba3-22c9ea79636a" (UID: "22f64ca3-f820-4609-aba3-22c9ea79636a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:19:09.620992 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.620811 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "22f64ca3-f820-4609-aba3-22c9ea79636a" (UID: "22f64ca3-f820-4609-aba3-22c9ea79636a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:19:09.621243 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.621183 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-data" (OuterVolumeSpecName: "istio-data") pod "22f64ca3-f820-4609-aba3-22c9ea79636a" (UID: "22f64ca3-f820-4609-aba3-22c9ea79636a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:19:09.622512 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.622484 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "22f64ca3-f820-4609-aba3-22c9ea79636a" (UID: "22f64ca3-f820-4609-aba3-22c9ea79636a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 13:19:09.622769 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.622740 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-token" (OuterVolumeSpecName: "istio-token") pod "22f64ca3-f820-4609-aba3-22c9ea79636a" (UID: "22f64ca3-f820-4609-aba3-22c9ea79636a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:19:09.622875 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.622830 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "22f64ca3-f820-4609-aba3-22c9ea79636a" (UID: "22f64ca3-f820-4609-aba3-22c9ea79636a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:19:09.623125 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.623104 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22f64ca3-f820-4609-aba3-22c9ea79636a-kube-api-access-kvhdk" (OuterVolumeSpecName: "kube-api-access-kvhdk") pod "22f64ca3-f820-4609-aba3-22c9ea79636a" (UID: "22f64ca3-f820-4609-aba3-22c9ea79636a"). InnerVolumeSpecName "kube-api-access-kvhdk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:19:09.679032 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.678997 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" Jul 21 13:19:09.721417 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.721375 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22f64ca3-f820-4609-aba3-22c9ea79636a-istiod-ca-cert\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:19:09.721417 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.721415 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-podinfo\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:19:09.721678 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.721429 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-token\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:19:09.721678 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.721442 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-credential-socket\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:19:09.721678 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.721456 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-envoy\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:19:09.721678 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.721471 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvhdk\" (UniqueName: \"kubernetes.io/projected/22f64ca3-f820-4609-aba3-22c9ea79636a-kube-api-access-kvhdk\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:19:09.721678 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.721484 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-workload-certs\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:19:09.721678 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.721496 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22f64ca3-f820-4609-aba3-22c9ea79636a-istio-data\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:19:09.838530 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.838317 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf"] Jul 21 13:19:09.841389 ip-10-0-133-39 kubenswrapper[2555]: W0721 13:19:09.841354 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceefb3e4_347d_4d93_8f31_f9212e0fa469.slice/crio-e9fa2d3569c71af4bf1b1c3169d6983216bb795e24bdb10e3af9e57978e07b74 WatchSource:0}: Error finding container e9fa2d3569c71af4bf1b1c3169d6983216bb795e24bdb10e3af9e57978e07b74: Status 404 returned error can't find the container with id e9fa2d3569c71af4bf1b1c3169d6983216bb795e24bdb10e3af9e57978e07b74 Jul 21 13:19:09.842853 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:09.842831 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 13:19:10.368862 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:10.368829 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" Jul 21 13:19:10.368862 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:10.368858 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq" event={"ID":"22f64ca3-f820-4609-aba3-22c9ea79636a","Type":"ContainerDied","Data":"1e06d83df0d3cf5ba1ed8119864b4241cdaa193c435a290b0c0ddb558942e8e8"} Jul 21 13:19:10.369134 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:10.368920 2555 scope.go:117] "RemoveContainer" containerID="76da034c4c76005645524299bf940e589a7ffe1de684217d8aa7d14f39f3821b" Jul 21 13:19:10.370377 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:10.370269 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" event={"ID":"ceefb3e4-347d-4d93-8f31-f9212e0fa469","Type":"ContainerStarted","Data":"e9fa2d3569c71af4bf1b1c3169d6983216bb795e24bdb10e3af9e57978e07b74"} Jul 21 13:19:10.392239 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:10.392196 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq"] Jul 21 13:19:10.396794 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:10.396758 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-823a18-openshift-default-6c78d87bbf-h85lq"] Jul 21 13:19:11.378788 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:11.378752 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" event={"ID":"ceefb3e4-347d-4d93-8f31-f9212e0fa469","Type":"ContainerStarted","Data":"526841c847a0675bd073cc6b6aa0127aa832045c5fd33fb9922f2a9160754ae3"} Jul 21 13:19:11.379300 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:11.378834 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" Jul 21 13:19:11.400273 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:11.400213 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" podStartSLOduration=2.6568837370000002 podStartE2EDuration="3.400194423s" totalImagesPullingTime="743.310686ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:19:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:19:11.399546703 +0000 UTC m=+2374.017839651" watchObservedRunningTime="2026-07-21 13:19:11.400194423 +0000 UTC m=+2374.018487370" Jul 21 13:19:11.928426 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:11.928387 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22f64ca3-f820-4609-aba3-22c9ea79636a" path="/var/lib/kubelet/pods/22f64ca3-f820-4609-aba3-22c9ea79636a/volumes" Jul 21 13:19:14.125254 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.125212 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl"] Jul 21 13:19:14.125735 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.125563 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22f64ca3-f820-4609-aba3-22c9ea79636a" containerName="istio-proxy" Jul 21 13:19:14.125735 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.125573 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="22f64ca3-f820-4609-aba3-22c9ea79636a" containerName="istio-proxy" Jul 21 13:19:14.125735 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.125607 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1624f09f-5534-4043-8472-1f09ab94b6c2" containerName="istio-proxy" Jul 21 13:19:14.125735 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.125613 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="1624f09f-5534-4043-8472-1f09ab94b6c2" containerName="istio-proxy" Jul 21 13:19:14.125735 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.125673 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="22f64ca3-f820-4609-aba3-22c9ea79636a" containerName="istio-proxy" Jul 21 13:19:14.125735 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.125681 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="22f64ca3-f820-4609-aba3-22c9ea79636a" containerName="istio-proxy" Jul 21 13:19:14.128546 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.128527 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.131991 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.131967 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-04ee34-openshift-default-dockercfg-qgw7m\"" Jul 21 13:19:14.155756 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.155726 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl"] Jul 21 13:19:14.260439 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.260404 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-credential-socket\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.260645 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.260457 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-workload-socket\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.260645 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.260517 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-istio-envoy\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.260645 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.260547 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a9f2127-e677-4200-bf72-b878828dde2c-istiod-ca-cert\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.260645 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.260571 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7228\" (UniqueName: \"kubernetes.io/projected/4a9f2127-e677-4200-bf72-b878828dde2c-kube-api-access-z7228\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.260645 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.260626 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-workload-certs\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.260828 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.260659 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a9f2127-e677-4200-bf72-b878828dde2c-istio-podinfo\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.260828 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.260731 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-istio-data\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.260828 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.260775 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a9f2127-e677-4200-bf72-b878828dde2c-istio-token\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.361998 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.361956 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-workload-socket\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.362229 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.362012 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-istio-envoy\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.362229 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.362043 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a9f2127-e677-4200-bf72-b878828dde2c-istiod-ca-cert\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.362229 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.362070 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7228\" (UniqueName: \"kubernetes.io/projected/4a9f2127-e677-4200-bf72-b878828dde2c-kube-api-access-z7228\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.362229 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.362101 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-workload-certs\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.362229 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.362166 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a9f2127-e677-4200-bf72-b878828dde2c-istio-podinfo\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.362229 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.362226 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-istio-data\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.362625 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.362255 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a9f2127-e677-4200-bf72-b878828dde2c-istio-token\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.362625 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.362301 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-credential-socket\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.362625 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.362397 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-workload-socket\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.362625 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.362571 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-workload-certs\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.362860 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.362795 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-istio-data\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.362919 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.362872 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a9f2127-e677-4200-bf72-b878828dde2c-istiod-ca-cert\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.362919 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.362885 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-credential-socket\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.364496 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.364472 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-istio-envoy\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.364884 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.364865 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a9f2127-e677-4200-bf72-b878828dde2c-istio-podinfo\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.373046 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.373017 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a9f2127-e677-4200-bf72-b878828dde2c-istio-token\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.373812 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.373792 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7228\" (UniqueName: \"kubernetes.io/projected/4a9f2127-e677-4200-bf72-b878828dde2c-kube-api-access-z7228\") pod \"e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.438796 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.438704 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:14.591643 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.591619 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl"] Jul 21 13:19:14.593841 ip-10-0-133-39 kubenswrapper[2555]: W0721 13:19:14.593818 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a9f2127_e677_4200_bf72_b878828dde2c.slice/crio-b8da3088b51a6d9be0f3fb8ed9b97a8a7b4d5cc65e70638fb4e698783bb452eb WatchSource:0}: Error finding container b8da3088b51a6d9be0f3fb8ed9b97a8a7b4d5cc65e70638fb4e698783bb452eb: Status 404 returned error can't find the container with id b8da3088b51a6d9be0f3fb8ed9b97a8a7b4d5cc65e70638fb4e698783bb452eb Jul 21 13:19:14.595886 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.595852 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:19:14.595978 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.595918 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:19:14.595978 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:14.595945 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:19:15.402253 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:15.402220 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" event={"ID":"4a9f2127-e677-4200-bf72-b878828dde2c","Type":"ContainerStarted","Data":"a41d0f39d3e8b4b628fbc4ad509b69e3a111c7534b140d3281cfdb693c6a6c34"} Jul 21 13:19:15.402674 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:15.402268 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" event={"ID":"4a9f2127-e677-4200-bf72-b878828dde2c","Type":"ContainerStarted","Data":"b8da3088b51a6d9be0f3fb8ed9b97a8a7b4d5cc65e70638fb4e698783bb452eb"} Jul 21 13:19:15.424122 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:15.424056 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" podStartSLOduration=1.424036767 podStartE2EDuration="1.424036767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:19:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:19:15.422985431 +0000 UTC m=+2378.041278377" watchObservedRunningTime="2026-07-21 13:19:15.424036767 +0000 UTC m=+2378.042329725" Jul 21 13:19:15.439885 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:15.439845 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:15.444951 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:15.444921 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:16.407256 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:16.407216 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:16.408389 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:16.408371 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:17.391323 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:17.391294 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" Jul 21 13:19:17.511125 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:17.511092 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74"] Jul 21 13:19:17.511521 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:17.511466 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22f64ca3-f820-4609-aba3-22c9ea79636a" containerName="istio-proxy" Jul 21 13:19:17.511521 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:17.511477 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="22f64ca3-f820-4609-aba3-22c9ea79636a" containerName="istio-proxy" Jul 21 13:19:17.514820 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:17.514795 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" Jul 21 13:19:17.517827 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:17.517805 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-04ee34-serving-cert\"" Jul 21 13:19:17.538942 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:17.538906 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74"] Jul 21 13:19:17.694976 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:17.694877 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf572\" (UniqueName: \"kubernetes.io/projected/ae612f51-585d-4bf0-8345-a550bcc33cef-kube-api-access-nf572\") pod \"maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74\" (UID: \"ae612f51-585d-4bf0-8345-a550bcc33cef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" Jul 21 13:19:17.694976 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:17.694921 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae612f51-585d-4bf0-8345-a550bcc33cef-maas-api-tls\") pod \"maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74\" (UID: \"ae612f51-585d-4bf0-8345-a550bcc33cef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" Jul 21 13:19:17.694976 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:17.694944 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae612f51-585d-4bf0-8345-a550bcc33cef-openshift-service-ca\") pod \"maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74\" (UID: \"ae612f51-585d-4bf0-8345-a550bcc33cef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" Jul 21 13:19:17.795482 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:17.795443 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf572\" (UniqueName: \"kubernetes.io/projected/ae612f51-585d-4bf0-8345-a550bcc33cef-kube-api-access-nf572\") pod \"maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74\" (UID: \"ae612f51-585d-4bf0-8345-a550bcc33cef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" Jul 21 13:19:17.795721 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:17.795491 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae612f51-585d-4bf0-8345-a550bcc33cef-maas-api-tls\") pod \"maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74\" (UID: \"ae612f51-585d-4bf0-8345-a550bcc33cef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" Jul 21 13:19:17.795721 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:17.795531 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae612f51-585d-4bf0-8345-a550bcc33cef-openshift-service-ca\") pod \"maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74\" (UID: \"ae612f51-585d-4bf0-8345-a550bcc33cef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" Jul 21 13:19:17.796271 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:17.796247 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae612f51-585d-4bf0-8345-a550bcc33cef-openshift-service-ca\") pod \"maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74\" (UID: \"ae612f51-585d-4bf0-8345-a550bcc33cef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" Jul 21 13:19:17.798231 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:17.798211 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae612f51-585d-4bf0-8345-a550bcc33cef-maas-api-tls\") pod \"maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74\" (UID: \"ae612f51-585d-4bf0-8345-a550bcc33cef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" Jul 21 13:19:17.804363 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:17.804332 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf572\" (UniqueName: \"kubernetes.io/projected/ae612f51-585d-4bf0-8345-a550bcc33cef-kube-api-access-nf572\") pod \"maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74\" (UID: \"ae612f51-585d-4bf0-8345-a550bcc33cef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" Jul 21 13:19:17.828038 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:17.828004 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" Jul 21 13:19:17.996647 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:17.994613 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74"] Jul 21 13:19:18.421518 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:18.421484 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" event={"ID":"ae612f51-585d-4bf0-8345-a550bcc33cef","Type":"ContainerStarted","Data":"0b4803487b26e0f531ba142a2ad94b33317d4de9f567cf26d2407d9d0fd251fa"} Jul 21 13:19:19.428713 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:19.428670 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" event={"ID":"ae612f51-585d-4bf0-8345-a550bcc33cef","Type":"ContainerStarted","Data":"ac7cd61feb4e8cac523e53b60b9e4638bde90aa526a1591490b472b008ab3f4c"} Jul 21 13:19:19.429110 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:19.428741 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" Jul 21 13:19:19.463582 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:19.463530 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" podStartSLOduration=1.6824593129999998 podStartE2EDuration="2.46351213s" totalImagesPullingTime="781.052817ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:19:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:19:19.45965263 +0000 UTC m=+2382.077945573" watchObservedRunningTime="2026-07-21 13:19:19.46351213 +0000 UTC m=+2382.081805127" Jul 21 13:19:25.445993 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:25.445959 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" Jul 21 13:19:31.408933 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:31.408899 2555 patch_prober.go:28] interesting pod/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 21 13:19:31.409458 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:31.408971 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" podUID="4a9f2127-e677-4200-bf72-b878828dde2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 21 13:19:31.497274 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:31.497238 2555 generic.go:356] "Generic (PLEG): container finished" podID="4a9f2127-e677-4200-bf72-b878828dde2c" containerID="a41d0f39d3e8b4b628fbc4ad509b69e3a111c7534b140d3281cfdb693c6a6c34" exitCode=0 Jul 21 13:19:31.497451 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:31.497313 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" event={"ID":"4a9f2127-e677-4200-bf72-b878828dde2c","Type":"ContainerDied","Data":"a41d0f39d3e8b4b628fbc4ad509b69e3a111c7534b140d3281cfdb693c6a6c34"} Jul 21 13:19:31.497777 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:31.497763 2555 scope.go:117] "RemoveContainer" containerID="a41d0f39d3e8b4b628fbc4ad509b69e3a111c7534b140d3281cfdb693c6a6c34" Jul 21 13:19:31.498836 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:31.498807 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:19:31.498899 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:31.498882 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:19:31.498945 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:31.498913 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:19:32.504314 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:32.504279 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" event={"ID":"4a9f2127-e677-4200-bf72-b878828dde2c","Type":"ContainerStarted","Data":"e41129cce73776420673f589eaa6c24101d894f3130cfc249e2b3cfb2f343058"} Jul 21 13:19:33.440428 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:33.439977 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:33.440428 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:33.440107 2555 patch_prober.go:28] interesting pod/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 21 13:19:33.440428 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:33.440164 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" podUID="4a9f2127-e677-4200-bf72-b878828dde2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 21 13:19:34.440441 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:34.440401 2555 patch_prober.go:28] interesting pod/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 21 13:19:34.441055 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:34.441023 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" podUID="4a9f2127-e677-4200-bf72-b878828dde2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 21 13:19:35.439375 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:35.439329 2555 patch_prober.go:28] interesting pod/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 21 13:19:35.439626 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:35.439400 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" podUID="4a9f2127-e677-4200-bf72-b878828dde2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 21 13:19:36.443563 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:36.443532 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:36.443976 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:36.443888 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:36.444529 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:36.444508 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:19:38.044038 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:38.044009 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 13:19:38.049044 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:38.049018 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 13:19:38.050255 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:38.050232 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 13:19:38.055683 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:38.055663 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 13:19:45.360558 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:45.360513 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf"] Jul 21 13:19:45.361150 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:45.360788 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" podUID="ceefb3e4-347d-4d93-8f31-f9212e0fa469" containerName="maas-api" containerID="cri-o://526841c847a0675bd073cc6b6aa0127aa832045c5fd33fb9922f2a9160754ae3" gracePeriod=30 Jul 21 13:19:45.580611 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:45.580569 2555 generic.go:356] "Generic (PLEG): container finished" podID="ceefb3e4-347d-4d93-8f31-f9212e0fa469" containerID="526841c847a0675bd073cc6b6aa0127aa832045c5fd33fb9922f2a9160754ae3" exitCode=0 Jul 21 13:19:45.580814 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:45.580629 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" event={"ID":"ceefb3e4-347d-4d93-8f31-f9212e0fa469","Type":"ContainerDied","Data":"526841c847a0675bd073cc6b6aa0127aa832045c5fd33fb9922f2a9160754ae3"} Jul 21 13:19:45.615490 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:45.615426 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" Jul 21 13:19:45.643106 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:45.643076 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceefb3e4-347d-4d93-8f31-f9212e0fa469-openshift-service-ca\") pod \"ceefb3e4-347d-4d93-8f31-f9212e0fa469\" (UID: \"ceefb3e4-347d-4d93-8f31-f9212e0fa469\") " Jul 21 13:19:45.643292 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:45.643123 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceefb3e4-347d-4d93-8f31-f9212e0fa469-maas-api-tls\") pod \"ceefb3e4-347d-4d93-8f31-f9212e0fa469\" (UID: \"ceefb3e4-347d-4d93-8f31-f9212e0fa469\") " Jul 21 13:19:45.643292 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:45.643172 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45sfq\" (UniqueName: \"kubernetes.io/projected/ceefb3e4-347d-4d93-8f31-f9212e0fa469-kube-api-access-45sfq\") pod \"ceefb3e4-347d-4d93-8f31-f9212e0fa469\" (UID: \"ceefb3e4-347d-4d93-8f31-f9212e0fa469\") " Jul 21 13:19:45.643522 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:45.643494 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceefb3e4-347d-4d93-8f31-f9212e0fa469-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ceefb3e4-347d-4d93-8f31-f9212e0fa469" (UID: "ceefb3e4-347d-4d93-8f31-f9212e0fa469"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:19:45.645408 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:45.645379 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceefb3e4-347d-4d93-8f31-f9212e0fa469-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ceefb3e4-347d-4d93-8f31-f9212e0fa469" (UID: "ceefb3e4-347d-4d93-8f31-f9212e0fa469"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:19:45.645537 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:45.645402 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ceefb3e4-347d-4d93-8f31-f9212e0fa469-kube-api-access-45sfq" (OuterVolumeSpecName: "kube-api-access-45sfq") pod "ceefb3e4-347d-4d93-8f31-f9212e0fa469" (UID: "ceefb3e4-347d-4d93-8f31-f9212e0fa469"). InnerVolumeSpecName "kube-api-access-45sfq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:19:45.743815 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:45.743784 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceefb3e4-347d-4d93-8f31-f9212e0fa469-openshift-service-ca\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:19:45.743815 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:45.743809 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceefb3e4-347d-4d93-8f31-f9212e0fa469-maas-api-tls\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:19:45.743815 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:45.743819 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45sfq\" (UniqueName: \"kubernetes.io/projected/ceefb3e4-347d-4d93-8f31-f9212e0fa469-kube-api-access-45sfq\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:19:46.587307 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:46.587276 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" Jul 21 13:19:46.587795 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:46.587310 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf" event={"ID":"ceefb3e4-347d-4d93-8f31-f9212e0fa469","Type":"ContainerDied","Data":"e9fa2d3569c71af4bf1b1c3169d6983216bb795e24bdb10e3af9e57978e07b74"} Jul 21 13:19:46.587795 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:46.587383 2555 scope.go:117] "RemoveContainer" containerID="526841c847a0675bd073cc6b6aa0127aa832045c5fd33fb9922f2a9160754ae3" Jul 21 13:19:46.607114 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:46.607082 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf"] Jul 21 13:19:46.618164 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:46.612628 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d921bc-866857c7cf-vklxf"] Jul 21 13:19:47.927392 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:19:47.927355 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ceefb3e4-347d-4d93-8f31-f9212e0fa469" path="/var/lib/kubelet/pods/ceefb3e4-347d-4d93-8f31-f9212e0fa469/volumes" Jul 21 13:20:14.033774 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.033737 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74"] Jul 21 13:20:14.034271 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.033995 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" podUID="ae612f51-585d-4bf0-8345-a550bcc33cef" containerName="maas-api" containerID="cri-o://ac7cd61feb4e8cac523e53b60b9e4638bde90aa526a1591490b472b008ab3f4c" gracePeriod=30 Jul 21 13:20:14.303509 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.303484 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" Jul 21 13:20:14.398587 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.398544 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae612f51-585d-4bf0-8345-a550bcc33cef-maas-api-tls\") pod \"ae612f51-585d-4bf0-8345-a550bcc33cef\" (UID: \"ae612f51-585d-4bf0-8345-a550bcc33cef\") " Jul 21 13:20:14.398814 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.398644 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nf572\" (UniqueName: \"kubernetes.io/projected/ae612f51-585d-4bf0-8345-a550bcc33cef-kube-api-access-nf572\") pod \"ae612f51-585d-4bf0-8345-a550bcc33cef\" (UID: \"ae612f51-585d-4bf0-8345-a550bcc33cef\") " Jul 21 13:20:14.398814 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.398747 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae612f51-585d-4bf0-8345-a550bcc33cef-openshift-service-ca\") pod \"ae612f51-585d-4bf0-8345-a550bcc33cef\" (UID: \"ae612f51-585d-4bf0-8345-a550bcc33cef\") " Jul 21 13:20:14.399115 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.399087 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae612f51-585d-4bf0-8345-a550bcc33cef-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ae612f51-585d-4bf0-8345-a550bcc33cef" (UID: "ae612f51-585d-4bf0-8345-a550bcc33cef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:20:14.400976 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.400948 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae612f51-585d-4bf0-8345-a550bcc33cef-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ae612f51-585d-4bf0-8345-a550bcc33cef" (UID: "ae612f51-585d-4bf0-8345-a550bcc33cef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:20:14.401080 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.400974 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae612f51-585d-4bf0-8345-a550bcc33cef-kube-api-access-nf572" (OuterVolumeSpecName: "kube-api-access-nf572") pod "ae612f51-585d-4bf0-8345-a550bcc33cef" (UID: "ae612f51-585d-4bf0-8345-a550bcc33cef"). InnerVolumeSpecName "kube-api-access-nf572". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:20:14.499634 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.499555 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae612f51-585d-4bf0-8345-a550bcc33cef-maas-api-tls\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:20:14.499634 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.499624 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nf572\" (UniqueName: \"kubernetes.io/projected/ae612f51-585d-4bf0-8345-a550bcc33cef-kube-api-access-nf572\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:20:14.499634 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.499639 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae612f51-585d-4bf0-8345-a550bcc33cef-openshift-service-ca\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:20:14.750355 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.750318 2555 generic.go:356] "Generic (PLEG): container finished" podID="ae612f51-585d-4bf0-8345-a550bcc33cef" containerID="ac7cd61feb4e8cac523e53b60b9e4638bde90aa526a1591490b472b008ab3f4c" exitCode=0 Jul 21 13:20:14.750540 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.750370 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" Jul 21 13:20:14.750540 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.750396 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" event={"ID":"ae612f51-585d-4bf0-8345-a550bcc33cef","Type":"ContainerDied","Data":"ac7cd61feb4e8cac523e53b60b9e4638bde90aa526a1591490b472b008ab3f4c"} Jul 21 13:20:14.750540 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.750450 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74" event={"ID":"ae612f51-585d-4bf0-8345-a550bcc33cef","Type":"ContainerDied","Data":"0b4803487b26e0f531ba142a2ad94b33317d4de9f567cf26d2407d9d0fd251fa"} Jul 21 13:20:14.750540 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.750475 2555 scope.go:117] "RemoveContainer" containerID="ac7cd61feb4e8cac523e53b60b9e4638bde90aa526a1591490b472b008ab3f4c" Jul 21 13:20:14.760670 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.760648 2555 scope.go:117] "RemoveContainer" containerID="ac7cd61feb4e8cac523e53b60b9e4638bde90aa526a1591490b472b008ab3f4c" Jul 21 13:20:14.761007 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:20:14.760980 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac7cd61feb4e8cac523e53b60b9e4638bde90aa526a1591490b472b008ab3f4c\": container with ID starting with ac7cd61feb4e8cac523e53b60b9e4638bde90aa526a1591490b472b008ab3f4c not found: ID does not exist" containerID="ac7cd61feb4e8cac523e53b60b9e4638bde90aa526a1591490b472b008ab3f4c" Jul 21 13:20:14.761102 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.761016 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac7cd61feb4e8cac523e53b60b9e4638bde90aa526a1591490b472b008ab3f4c"} err="failed to get container status \"ac7cd61feb4e8cac523e53b60b9e4638bde90aa526a1591490b472b008ab3f4c\": rpc error: code = NotFound desc = could not find container \"ac7cd61feb4e8cac523e53b60b9e4638bde90aa526a1591490b472b008ab3f4c\": container with ID starting with ac7cd61feb4e8cac523e53b60b9e4638bde90aa526a1591490b472b008ab3f4c not found: ID does not exist" Jul 21 13:20:14.776888 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.776855 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74"] Jul 21 13:20:14.782055 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:14.782027 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-04ee34-7b8d7594cb-pdj74"] Jul 21 13:20:15.926651 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:15.926617 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae612f51-585d-4bf0-8345-a550bcc33cef" path="/var/lib/kubelet/pods/ae612f51-585d-4bf0-8345-a550bcc33cef/volumes" Jul 21 13:20:25.995423 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:25.995381 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl"] Jul 21 13:20:25.995949 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:25.995693 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" podUID="4a9f2127-e677-4200-bf72-b878828dde2c" containerName="istio-proxy" containerID="cri-o://e41129cce73776420673f589eaa6c24101d894f3130cfc249e2b3cfb2f343058" gracePeriod=30 Jul 21 13:20:30.295912 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.295880 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb"] Jul 21 13:20:30.296435 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.296416 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceefb3e4-347d-4d93-8f31-f9212e0fa469" containerName="maas-api" Jul 21 13:20:30.296516 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.296439 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceefb3e4-347d-4d93-8f31-f9212e0fa469" containerName="maas-api" Jul 21 13:20:30.296516 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.296464 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae612f51-585d-4bf0-8345-a550bcc33cef" containerName="maas-api" Jul 21 13:20:30.296516 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.296473 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae612f51-585d-4bf0-8345-a550bcc33cef" containerName="maas-api" Jul 21 13:20:30.296711 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.296566 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae612f51-585d-4bf0-8345-a550bcc33cef" containerName="maas-api" Jul 21 13:20:30.296711 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.296580 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceefb3e4-347d-4d93-8f31-f9212e0fa469" containerName="maas-api" Jul 21 13:20:30.299572 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.299550 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" Jul 21 13:20:30.302888 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.302865 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-njpqk\"" Jul 21 13:20:30.303045 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.302917 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 13:20:30.303045 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.302930 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-b3a013-serving-cert\"" Jul 21 13:20:30.303045 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.302865 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 13:20:30.315144 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.315114 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb"] Jul 21 13:20:30.339603 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.339568 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7091c316-c17d-4263-99aa-8f4741dcf257-maas-api-tls\") pod \"maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb\" (UID: \"7091c316-c17d-4263-99aa-8f4741dcf257\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" Jul 21 13:20:30.339794 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.339663 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7091c316-c17d-4263-99aa-8f4741dcf257-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb\" (UID: \"7091c316-c17d-4263-99aa-8f4741dcf257\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" Jul 21 13:20:30.339794 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.339704 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-967wp\" (UniqueName: \"kubernetes.io/projected/7091c316-c17d-4263-99aa-8f4741dcf257-kube-api-access-967wp\") pod \"maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb\" (UID: \"7091c316-c17d-4263-99aa-8f4741dcf257\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" Jul 21 13:20:30.441068 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.441032 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-967wp\" (UniqueName: \"kubernetes.io/projected/7091c316-c17d-4263-99aa-8f4741dcf257-kube-api-access-967wp\") pod \"maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb\" (UID: \"7091c316-c17d-4263-99aa-8f4741dcf257\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" Jul 21 13:20:30.441277 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.441098 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7091c316-c17d-4263-99aa-8f4741dcf257-maas-api-tls\") pod \"maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb\" (UID: \"7091c316-c17d-4263-99aa-8f4741dcf257\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" Jul 21 13:20:30.441277 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.441133 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7091c316-c17d-4263-99aa-8f4741dcf257-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb\" (UID: \"7091c316-c17d-4263-99aa-8f4741dcf257\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" Jul 21 13:20:30.441277 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:20:30.441241 2555 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-serving-cert: secret "maas-api-e2e-shared-a-b3a013-serving-cert" not found Jul 21 13:20:30.441447 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:20:30.441314 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7091c316-c17d-4263-99aa-8f4741dcf257-maas-api-tls podName:7091c316-c17d-4263-99aa-8f4741dcf257 nodeName:}" failed. No retries permitted until 2026-07-21 13:20:30.941292741 +0000 UTC m=+2453.559585667 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7091c316-c17d-4263-99aa-8f4741dcf257-maas-api-tls") pod "maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" (UID: "7091c316-c17d-4263-99aa-8f4741dcf257") : secret "maas-api-e2e-shared-a-b3a013-serving-cert" not found Jul 21 13:20:30.441872 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.441855 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7091c316-c17d-4263-99aa-8f4741dcf257-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb\" (UID: \"7091c316-c17d-4263-99aa-8f4741dcf257\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" Jul 21 13:20:30.454100 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.454073 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-967wp\" (UniqueName: \"kubernetes.io/projected/7091c316-c17d-4263-99aa-8f4741dcf257-kube-api-access-967wp\") pod \"maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb\" (UID: \"7091c316-c17d-4263-99aa-8f4741dcf257\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" Jul 21 13:20:30.945468 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.945433 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7091c316-c17d-4263-99aa-8f4741dcf257-maas-api-tls\") pod \"maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb\" (UID: \"7091c316-c17d-4263-99aa-8f4741dcf257\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" Jul 21 13:20:30.948164 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:30.948134 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7091c316-c17d-4263-99aa-8f4741dcf257-maas-api-tls\") pod \"maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb\" (UID: \"7091c316-c17d-4263-99aa-8f4741dcf257\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" Jul 21 13:20:31.210888 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.210809 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" Jul 21 13:20:31.253043 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.253007 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:20:31.349225 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.349197 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-workload-socket\") pod \"4a9f2127-e677-4200-bf72-b878828dde2c\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " Jul 21 13:20:31.349694 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.349233 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a9f2127-e677-4200-bf72-b878828dde2c-istio-token\") pod \"4a9f2127-e677-4200-bf72-b878828dde2c\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " Jul 21 13:20:31.349694 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.349268 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-istio-data\") pod \"4a9f2127-e677-4200-bf72-b878828dde2c\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " Jul 21 13:20:31.349694 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.349341 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-istio-envoy\") pod \"4a9f2127-e677-4200-bf72-b878828dde2c\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " Jul 21 13:20:31.349694 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.349372 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a9f2127-e677-4200-bf72-b878828dde2c-istio-podinfo\") pod \"4a9f2127-e677-4200-bf72-b878828dde2c\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " Jul 21 13:20:31.349694 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.349401 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a9f2127-e677-4200-bf72-b878828dde2c-istiod-ca-cert\") pod \"4a9f2127-e677-4200-bf72-b878828dde2c\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " Jul 21 13:20:31.349694 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.349426 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7228\" (UniqueName: \"kubernetes.io/projected/4a9f2127-e677-4200-bf72-b878828dde2c-kube-api-access-z7228\") pod \"4a9f2127-e677-4200-bf72-b878828dde2c\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " Jul 21 13:20:31.349694 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.349458 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-workload-certs\") pod \"4a9f2127-e677-4200-bf72-b878828dde2c\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " Jul 21 13:20:31.349694 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.349496 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4a9f2127-e677-4200-bf72-b878828dde2c" (UID: "4a9f2127-e677-4200-bf72-b878828dde2c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:20:31.349694 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.349533 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-credential-socket\") pod \"4a9f2127-e677-4200-bf72-b878828dde2c\" (UID: \"4a9f2127-e677-4200-bf72-b878828dde2c\") " Jul 21 13:20:31.352623 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.350801 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a9f2127-e677-4200-bf72-b878828dde2c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4a9f2127-e677-4200-bf72-b878828dde2c" (UID: "4a9f2127-e677-4200-bf72-b878828dde2c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:20:31.352623 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.351211 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4a9f2127-e677-4200-bf72-b878828dde2c" (UID: "4a9f2127-e677-4200-bf72-b878828dde2c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:20:31.352623 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.351503 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-workload-socket\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:20:31.352623 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.351524 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a9f2127-e677-4200-bf72-b878828dde2c-istiod-ca-cert\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:20:31.352623 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.351546 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-workload-certs\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:20:31.352623 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.352265 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-istio-data" (OuterVolumeSpecName: "istio-data") pod "4a9f2127-e677-4200-bf72-b878828dde2c" (UID: "4a9f2127-e677-4200-bf72-b878828dde2c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:20:31.353212 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.353181 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4a9f2127-e677-4200-bf72-b878828dde2c" (UID: "4a9f2127-e677-4200-bf72-b878828dde2c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:20:31.356586 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.356551 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a9f2127-e677-4200-bf72-b878828dde2c-kube-api-access-z7228" (OuterVolumeSpecName: "kube-api-access-z7228") pod "4a9f2127-e677-4200-bf72-b878828dde2c" (UID: "4a9f2127-e677-4200-bf72-b878828dde2c"). InnerVolumeSpecName "kube-api-access-z7228". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:20:31.356779 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.356569 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a9f2127-e677-4200-bf72-b878828dde2c-istio-token" (OuterVolumeSpecName: "istio-token") pod "4a9f2127-e677-4200-bf72-b878828dde2c" (UID: "4a9f2127-e677-4200-bf72-b878828dde2c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:20:31.356882 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.356584 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4a9f2127-e677-4200-bf72-b878828dde2c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4a9f2127-e677-4200-bf72-b878828dde2c" (UID: "4a9f2127-e677-4200-bf72-b878828dde2c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 13:20:31.364849 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.364783 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4a9f2127-e677-4200-bf72-b878828dde2c" (UID: "4a9f2127-e677-4200-bf72-b878828dde2c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:20:31.367950 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.367926 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb"] Jul 21 13:20:31.370031 ip-10-0-133-39 kubenswrapper[2555]: W0721 13:20:31.370001 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7091c316_c17d_4263_99aa_8f4741dcf257.slice/crio-9110a8953901f218be393168ecdcce99433ad7b0e3d2baf72172961d24434cc0 WatchSource:0}: Error finding container 9110a8953901f218be393168ecdcce99433ad7b0e3d2baf72172961d24434cc0: Status 404 returned error can't find the container with id 9110a8953901f218be393168ecdcce99433ad7b0e3d2baf72172961d24434cc0 Jul 21 13:20:31.451970 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.451930 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-credential-socket\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:20:31.451970 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.451962 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a9f2127-e677-4200-bf72-b878828dde2c-istio-token\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:20:31.451970 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.451977 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-istio-data\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:20:31.452225 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.451988 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a9f2127-e677-4200-bf72-b878828dde2c-istio-envoy\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:20:31.452225 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.451999 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a9f2127-e677-4200-bf72-b878828dde2c-istio-podinfo\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:20:31.452225 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.452010 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7228\" (UniqueName: \"kubernetes.io/projected/4a9f2127-e677-4200-bf72-b878828dde2c-kube-api-access-z7228\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:20:31.853686 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.853649 2555 generic.go:356] "Generic (PLEG): container finished" podID="4a9f2127-e677-4200-bf72-b878828dde2c" containerID="e41129cce73776420673f589eaa6c24101d894f3130cfc249e2b3cfb2f343058" exitCode=0 Jul 21 13:20:31.853873 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.853709 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" Jul 21 13:20:31.853873 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.853731 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" event={"ID":"4a9f2127-e677-4200-bf72-b878828dde2c","Type":"ContainerDied","Data":"e41129cce73776420673f589eaa6c24101d894f3130cfc249e2b3cfb2f343058"} Jul 21 13:20:31.853873 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.853799 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl" event={"ID":"4a9f2127-e677-4200-bf72-b878828dde2c","Type":"ContainerDied","Data":"b8da3088b51a6d9be0f3fb8ed9b97a8a7b4d5cc65e70638fb4e698783bb452eb"} Jul 21 13:20:31.853873 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.853824 2555 scope.go:117] "RemoveContainer" containerID="e41129cce73776420673f589eaa6c24101d894f3130cfc249e2b3cfb2f343058" Jul 21 13:20:31.855549 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.855524 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" event={"ID":"7091c316-c17d-4263-99aa-8f4741dcf257","Type":"ContainerStarted","Data":"2b026b6558c2e1016d7f1cefc7d5af47112dc1e1f09f4cb230acbb3f5cdf2751"} Jul 21 13:20:31.855705 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.855577 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" event={"ID":"7091c316-c17d-4263-99aa-8f4741dcf257","Type":"ContainerStarted","Data":"9110a8953901f218be393168ecdcce99433ad7b0e3d2baf72172961d24434cc0"} Jul 21 13:20:31.855705 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.855637 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" Jul 21 13:20:31.864189 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.864171 2555 scope.go:117] "RemoveContainer" containerID="a41d0f39d3e8b4b628fbc4ad509b69e3a111c7534b140d3281cfdb693c6a6c34" Jul 21 13:20:31.873821 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.873800 2555 scope.go:117] "RemoveContainer" containerID="e41129cce73776420673f589eaa6c24101d894f3130cfc249e2b3cfb2f343058" Jul 21 13:20:31.874133 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:20:31.874109 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e41129cce73776420673f589eaa6c24101d894f3130cfc249e2b3cfb2f343058\": container with ID starting with e41129cce73776420673f589eaa6c24101d894f3130cfc249e2b3cfb2f343058 not found: ID does not exist" containerID="e41129cce73776420673f589eaa6c24101d894f3130cfc249e2b3cfb2f343058" Jul 21 13:20:31.874210 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.874141 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e41129cce73776420673f589eaa6c24101d894f3130cfc249e2b3cfb2f343058"} err="failed to get container status \"e41129cce73776420673f589eaa6c24101d894f3130cfc249e2b3cfb2f343058\": rpc error: code = NotFound desc = could not find container \"e41129cce73776420673f589eaa6c24101d894f3130cfc249e2b3cfb2f343058\": container with ID starting with e41129cce73776420673f589eaa6c24101d894f3130cfc249e2b3cfb2f343058 not found: ID does not exist" Jul 21 13:20:31.874210 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.874158 2555 scope.go:117] "RemoveContainer" containerID="a41d0f39d3e8b4b628fbc4ad509b69e3a111c7534b140d3281cfdb693c6a6c34" Jul 21 13:20:31.874415 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:20:31.874396 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a41d0f39d3e8b4b628fbc4ad509b69e3a111c7534b140d3281cfdb693c6a6c34\": container with ID starting with a41d0f39d3e8b4b628fbc4ad509b69e3a111c7534b140d3281cfdb693c6a6c34 not found: ID does not exist" containerID="a41d0f39d3e8b4b628fbc4ad509b69e3a111c7534b140d3281cfdb693c6a6c34" Jul 21 13:20:31.874479 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.874426 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a41d0f39d3e8b4b628fbc4ad509b69e3a111c7534b140d3281cfdb693c6a6c34"} err="failed to get container status \"a41d0f39d3e8b4b628fbc4ad509b69e3a111c7534b140d3281cfdb693c6a6c34\": rpc error: code = NotFound desc = could not find container \"a41d0f39d3e8b4b628fbc4ad509b69e3a111c7534b140d3281cfdb693c6a6c34\": container with ID starting with a41d0f39d3e8b4b628fbc4ad509b69e3a111c7534b140d3281cfdb693c6a6c34 not found: ID does not exist" Jul 21 13:20:31.878951 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.878903 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" podStartSLOduration=1.5106953779999999 podStartE2EDuration="1.878885819s" totalImagesPullingTime="368.190441ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:20:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:20:31.877273999 +0000 UTC m=+2454.495566947" watchObservedRunningTime="2026-07-21 13:20:31.878885819 +0000 UTC m=+2454.497178823" Jul 21 13:20:31.896342 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.896307 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl"] Jul 21 13:20:31.900462 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.900423 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-04ee34-openshift-default-6c7c7bf5bc-9dgsl"] Jul 21 13:20:31.927428 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:31.927395 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a9f2127-e677-4200-bf72-b878828dde2c" path="/var/lib/kubelet/pods/4a9f2127-e677-4200-bf72-b878828dde2c/volumes" Jul 21 13:20:35.230314 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.230279 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt"] Jul 21 13:20:35.230839 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.230819 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a9f2127-e677-4200-bf72-b878828dde2c" containerName="istio-proxy" Jul 21 13:20:35.230917 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.230843 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a9f2127-e677-4200-bf72-b878828dde2c" containerName="istio-proxy" Jul 21 13:20:35.230991 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.230979 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a9f2127-e677-4200-bf72-b878828dde2c" containerName="istio-proxy" Jul 21 13:20:35.231041 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.230997 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a9f2127-e677-4200-bf72-b878828dde2c" containerName="istio-proxy" Jul 21 13:20:35.235966 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.235938 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.238126 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.238104 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-cfed01-openshift-default-dockercfg-p6sbk\"" Jul 21 13:20:35.244943 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.244916 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt"] Jul 21 13:20:35.286852 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.286817 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-token\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.286852 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.286857 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-envoy\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.287084 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.286878 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-workload-certs\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.287084 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.286895 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jldjq\" (UniqueName: \"kubernetes.io/projected/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-kube-api-access-jldjq\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.287084 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.286931 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-credential-socket\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.287084 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.286961 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-podinfo\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.287084 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.286978 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istiod-ca-cert\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.287084 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.287032 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-data\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.287084 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.287053 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-workload-socket\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.387445 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.387413 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-podinfo\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.387445 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.387448 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istiod-ca-cert\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.387699 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.387500 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-data\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.387699 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.387524 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-workload-socket\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.387699 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.387560 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-token\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.387699 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.387589 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-envoy\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.387932 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.387901 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-workload-certs\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.388007 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.387977 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jldjq\" (UniqueName: \"kubernetes.io/projected/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-kube-api-access-jldjq\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.388068 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.388033 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-credential-socket\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.388575 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.388291 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-workload-socket\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.388575 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.388337 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istiod-ca-cert\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.388575 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.388513 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-data\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.388575 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.388553 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-credential-socket\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.388906 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.388668 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-workload-certs\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.390150 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.390121 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-envoy\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.390347 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.390326 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-podinfo\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.397157 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.397127 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jldjq\" (UniqueName: \"kubernetes.io/projected/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-kube-api-access-jldjq\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.397273 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.397191 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-token\") pod \"e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.548414 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.548315 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:35.725319 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.725198 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt"] Jul 21 13:20:35.732436 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.732388 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:20:35.732548 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.732478 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:20:35.732548 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.732515 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:20:35.879585 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.879531 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" event={"ID":"06adbbc7-a40e-48c5-bd70-a93ec25e61ee","Type":"ContainerStarted","Data":"b34f1fb52c37c82deaa5b51aa6fa588da43e4a3a548c649a56c54893b6c86a55"} Jul 21 13:20:35.879780 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.879611 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" event={"ID":"06adbbc7-a40e-48c5-bd70-a93ec25e61ee","Type":"ContainerStarted","Data":"0ea9cd200aacd20b622fe9c70468aa5494a02885e8f300027e0f0f789a974cf8"} Jul 21 13:20:35.910758 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:35.910703 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" podStartSLOduration=0.910680497 podStartE2EDuration="910.680497ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:20:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:20:35.909546177 +0000 UTC m=+2458.527839130" watchObservedRunningTime="2026-07-21 13:20:35.910680497 +0000 UTC m=+2458.528973445" Jul 21 13:20:36.549476 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:36.549434 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:36.554452 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:36.554424 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:36.885169 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:36.885129 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:36.886347 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:36.886327 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:37.868187 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:37.868160 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" Jul 21 13:20:38.041431 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:38.041393 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2"] Jul 21 13:20:38.041944 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:38.041923 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a9f2127-e677-4200-bf72-b878828dde2c" containerName="istio-proxy" Jul 21 13:20:38.041944 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:38.041945 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a9f2127-e677-4200-bf72-b878828dde2c" containerName="istio-proxy" Jul 21 13:20:38.045104 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:38.045083 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" Jul 21 13:20:38.047292 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:38.047256 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-cfed01-serving-cert\"" Jul 21 13:20:38.058151 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:38.058119 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2"] Jul 21 13:20:38.116276 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:38.116240 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dwl7\" (UniqueName: \"kubernetes.io/projected/713ca469-cb40-47b0-a3ce-02c2c7bdf021-kube-api-access-5dwl7\") pod \"maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2\" (UID: \"713ca469-cb40-47b0-a3ce-02c2c7bdf021\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" Jul 21 13:20:38.116496 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:38.116299 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/713ca469-cb40-47b0-a3ce-02c2c7bdf021-maas-api-tls\") pod \"maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2\" (UID: \"713ca469-cb40-47b0-a3ce-02c2c7bdf021\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" Jul 21 13:20:38.116496 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:38.116404 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/713ca469-cb40-47b0-a3ce-02c2c7bdf021-openshift-service-ca\") pod \"maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2\" (UID: \"713ca469-cb40-47b0-a3ce-02c2c7bdf021\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" Jul 21 13:20:38.217506 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:38.217469 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dwl7\" (UniqueName: \"kubernetes.io/projected/713ca469-cb40-47b0-a3ce-02c2c7bdf021-kube-api-access-5dwl7\") pod \"maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2\" (UID: \"713ca469-cb40-47b0-a3ce-02c2c7bdf021\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" Jul 21 13:20:38.217728 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:38.217521 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/713ca469-cb40-47b0-a3ce-02c2c7bdf021-maas-api-tls\") pod \"maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2\" (UID: \"713ca469-cb40-47b0-a3ce-02c2c7bdf021\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" Jul 21 13:20:38.217728 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:38.217569 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/713ca469-cb40-47b0-a3ce-02c2c7bdf021-openshift-service-ca\") pod \"maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2\" (UID: \"713ca469-cb40-47b0-a3ce-02c2c7bdf021\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" Jul 21 13:20:38.218257 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:38.218235 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/713ca469-cb40-47b0-a3ce-02c2c7bdf021-openshift-service-ca\") pod \"maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2\" (UID: \"713ca469-cb40-47b0-a3ce-02c2c7bdf021\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" Jul 21 13:20:38.220332 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:38.220302 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/713ca469-cb40-47b0-a3ce-02c2c7bdf021-maas-api-tls\") pod \"maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2\" (UID: \"713ca469-cb40-47b0-a3ce-02c2c7bdf021\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" Jul 21 13:20:38.227184 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:38.227158 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dwl7\" (UniqueName: \"kubernetes.io/projected/713ca469-cb40-47b0-a3ce-02c2c7bdf021-kube-api-access-5dwl7\") pod \"maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2\" (UID: \"713ca469-cb40-47b0-a3ce-02c2c7bdf021\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" Jul 21 13:20:38.356878 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:38.356844 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" Jul 21 13:20:38.500435 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:38.500410 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2"] Jul 21 13:20:38.502280 ip-10-0-133-39 kubenswrapper[2555]: W0721 13:20:38.502245 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod713ca469_cb40_47b0_a3ce_02c2c7bdf021.slice/crio-5b70ef867c6fc136136980833ec4c7d7f70a84868e59bf2549fc07c74854062a WatchSource:0}: Error finding container 5b70ef867c6fc136136980833ec4c7d7f70a84868e59bf2549fc07c74854062a: Status 404 returned error can't find the container with id 5b70ef867c6fc136136980833ec4c7d7f70a84868e59bf2549fc07c74854062a Jul 21 13:20:38.898065 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:38.898031 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" event={"ID":"713ca469-cb40-47b0-a3ce-02c2c7bdf021","Type":"ContainerStarted","Data":"5b70ef867c6fc136136980833ec4c7d7f70a84868e59bf2549fc07c74854062a"} Jul 21 13:20:39.904579 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:39.904544 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" event={"ID":"713ca469-cb40-47b0-a3ce-02c2c7bdf021","Type":"ContainerStarted","Data":"eb401e20c8c312545721408a4de3d21127b6849a75b3e76b89b101920bbd5676"} Jul 21 13:20:39.905020 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:39.904649 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" Jul 21 13:20:39.926351 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:39.926297 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" podStartSLOduration=1.4680570130000001 podStartE2EDuration="1.926280891s" totalImagesPullingTime="458.223878ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:20:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:20:39.923745525 +0000 UTC m=+2462.542038474" watchObservedRunningTime="2026-07-21 13:20:39.926280891 +0000 UTC m=+2462.544573869" Jul 21 13:20:45.915574 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:45.915545 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" Jul 21 13:20:48.973614 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:48.973514 2555 generic.go:356] "Generic (PLEG): container finished" podID="06adbbc7-a40e-48c5-bd70-a93ec25e61ee" containerID="b34f1fb52c37c82deaa5b51aa6fa588da43e4a3a548c649a56c54893b6c86a55" exitCode=0 Jul 21 13:20:48.973614 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:48.973549 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" event={"ID":"06adbbc7-a40e-48c5-bd70-a93ec25e61ee","Type":"ContainerDied","Data":"b34f1fb52c37c82deaa5b51aa6fa588da43e4a3a548c649a56c54893b6c86a55"} Jul 21 13:20:48.974116 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:48.973931 2555 scope.go:117] "RemoveContainer" containerID="b34f1fb52c37c82deaa5b51aa6fa588da43e4a3a548c649a56c54893b6c86a55" Jul 21 13:20:48.974917 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:48.974888 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:20:48.974984 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:48.974963 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:20:48.975021 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:48.974998 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 13:20:49.980767 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:49.980735 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" event={"ID":"06adbbc7-a40e-48c5-bd70-a93ec25e61ee","Type":"ContainerStarted","Data":"e0c86f64c65cfa304facb53b973bb0ba27b2e6147ffbc8eab7794c6c65d1b9a7"} Jul 21 13:20:50.548616 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:50.548561 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:50.550045 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:50.550016 2555 patch_prober.go:28] interesting pod/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 21 13:20:50.550194 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:50.550064 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" podUID="06adbbc7-a40e-48c5-bd70-a93ec25e61ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 21 13:20:51.549000 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:51.548962 2555 patch_prober.go:28] interesting pod/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 21 13:20:51.549381 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:51.549026 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" podUID="06adbbc7-a40e-48c5-bd70-a93ec25e61ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 21 13:20:51.885275 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:51.885230 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:52.549390 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:52.549357 2555 patch_prober.go:28] interesting pod/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 21 13:20:52.549844 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:52.549416 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" podUID="06adbbc7-a40e-48c5-bd70-a93ec25e61ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 21 13:20:53.548860 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:53.548820 2555 patch_prober.go:28] interesting pod/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 21 13:20:53.549043 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:53.548874 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" podUID="06adbbc7-a40e-48c5-bd70-a93ec25e61ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 21 13:20:54.553226 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:54.553195 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:20:54.554250 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:20:54.554230 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:24:38.082755 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:24:38.082685 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 13:24:38.087944 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:24:38.087922 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 13:24:38.090840 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:24:38.090820 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 13:24:38.095644 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:24:38.095626 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ntfk8_313167cc-d4ec-4ab0-9f12-b66389ad21fb/ovn-acl-logging/0.log" Jul 21 13:26:48.253902 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.253865 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb"] Jul 21 13:26:48.254462 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.254085 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" podUID="7091c316-c17d-4263-99aa-8f4741dcf257" containerName="maas-api" containerID="cri-o://2b026b6558c2e1016d7f1cefc7d5af47112dc1e1f09f4cb230acbb3f5cdf2751" gracePeriod=30 Jul 21 13:26:48.522031 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.521997 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" Jul 21 13:26:48.599058 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.599015 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7091c316-c17d-4263-99aa-8f4741dcf257-openshift-service-ca\") pod \"7091c316-c17d-4263-99aa-8f4741dcf257\" (UID: \"7091c316-c17d-4263-99aa-8f4741dcf257\") " Jul 21 13:26:48.599267 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.599103 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7091c316-c17d-4263-99aa-8f4741dcf257-maas-api-tls\") pod \"7091c316-c17d-4263-99aa-8f4741dcf257\" (UID: \"7091c316-c17d-4263-99aa-8f4741dcf257\") " Jul 21 13:26:48.599267 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.599130 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-967wp\" (UniqueName: \"kubernetes.io/projected/7091c316-c17d-4263-99aa-8f4741dcf257-kube-api-access-967wp\") pod \"7091c316-c17d-4263-99aa-8f4741dcf257\" (UID: \"7091c316-c17d-4263-99aa-8f4741dcf257\") " Jul 21 13:26:48.599492 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.599469 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7091c316-c17d-4263-99aa-8f4741dcf257-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7091c316-c17d-4263-99aa-8f4741dcf257" (UID: "7091c316-c17d-4263-99aa-8f4741dcf257"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:26:48.601462 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.601431 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7091c316-c17d-4263-99aa-8f4741dcf257-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7091c316-c17d-4263-99aa-8f4741dcf257" (UID: "7091c316-c17d-4263-99aa-8f4741dcf257"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:26:48.601563 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.601468 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7091c316-c17d-4263-99aa-8f4741dcf257-kube-api-access-967wp" (OuterVolumeSpecName: "kube-api-access-967wp") pod "7091c316-c17d-4263-99aa-8f4741dcf257" (UID: "7091c316-c17d-4263-99aa-8f4741dcf257"). InnerVolumeSpecName "kube-api-access-967wp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:26:48.700726 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.700693 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-967wp\" (UniqueName: \"kubernetes.io/projected/7091c316-c17d-4263-99aa-8f4741dcf257-kube-api-access-967wp\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:26:48.700726 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.700720 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7091c316-c17d-4263-99aa-8f4741dcf257-openshift-service-ca\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:26:48.700726 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.700731 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7091c316-c17d-4263-99aa-8f4741dcf257-maas-api-tls\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:26:48.962924 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.962892 2555 generic.go:356] "Generic (PLEG): container finished" podID="7091c316-c17d-4263-99aa-8f4741dcf257" containerID="2b026b6558c2e1016d7f1cefc7d5af47112dc1e1f09f4cb230acbb3f5cdf2751" exitCode=0 Jul 21 13:26:48.963114 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.962941 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" Jul 21 13:26:48.963114 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.962939 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" event={"ID":"7091c316-c17d-4263-99aa-8f4741dcf257","Type":"ContainerDied","Data":"2b026b6558c2e1016d7f1cefc7d5af47112dc1e1f09f4cb230acbb3f5cdf2751"} Jul 21 13:26:48.963114 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.963080 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb" event={"ID":"7091c316-c17d-4263-99aa-8f4741dcf257","Type":"ContainerDied","Data":"9110a8953901f218be393168ecdcce99433ad7b0e3d2baf72172961d24434cc0"} Jul 21 13:26:48.963114 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.963111 2555 scope.go:117] "RemoveContainer" containerID="2b026b6558c2e1016d7f1cefc7d5af47112dc1e1f09f4cb230acbb3f5cdf2751" Jul 21 13:26:48.973125 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.973105 2555 scope.go:117] "RemoveContainer" containerID="2b026b6558c2e1016d7f1cefc7d5af47112dc1e1f09f4cb230acbb3f5cdf2751" Jul 21 13:26:48.973411 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:26:48.973390 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b026b6558c2e1016d7f1cefc7d5af47112dc1e1f09f4cb230acbb3f5cdf2751\": container with ID starting with 2b026b6558c2e1016d7f1cefc7d5af47112dc1e1f09f4cb230acbb3f5cdf2751 not found: ID does not exist" containerID="2b026b6558c2e1016d7f1cefc7d5af47112dc1e1f09f4cb230acbb3f5cdf2751" Jul 21 13:26:48.973494 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.973421 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b026b6558c2e1016d7f1cefc7d5af47112dc1e1f09f4cb230acbb3f5cdf2751"} err="failed to get container status \"2b026b6558c2e1016d7f1cefc7d5af47112dc1e1f09f4cb230acbb3f5cdf2751\": rpc error: code = NotFound desc = could not find container \"2b026b6558c2e1016d7f1cefc7d5af47112dc1e1f09f4cb230acbb3f5cdf2751\": container with ID starting with 2b026b6558c2e1016d7f1cefc7d5af47112dc1e1f09f4cb230acbb3f5cdf2751 not found: ID does not exist" Jul 21 13:26:48.986052 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.986018 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb"] Jul 21 13:26:48.990842 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:48.990807 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b3a013-7fc864fd5-q68rb"] Jul 21 13:26:49.926368 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:26:49.926332 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7091c316-c17d-4263-99aa-8f4741dcf257" path="/var/lib/kubelet/pods/7091c316-c17d-4263-99aa-8f4741dcf257/volumes" Jul 21 13:27:11.390585 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:11.390551 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2"] Jul 21 13:27:11.391154 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:11.390802 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" podUID="713ca469-cb40-47b0-a3ce-02c2c7bdf021" containerName="maas-api" containerID="cri-o://eb401e20c8c312545721408a4de3d21127b6849a75b3e76b89b101920bbd5676" gracePeriod=30 Jul 21 13:27:11.659823 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:11.659797 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" Jul 21 13:27:11.811872 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:11.811832 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dwl7\" (UniqueName: \"kubernetes.io/projected/713ca469-cb40-47b0-a3ce-02c2c7bdf021-kube-api-access-5dwl7\") pod \"713ca469-cb40-47b0-a3ce-02c2c7bdf021\" (UID: \"713ca469-cb40-47b0-a3ce-02c2c7bdf021\") " Jul 21 13:27:11.812085 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:11.811941 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/713ca469-cb40-47b0-a3ce-02c2c7bdf021-maas-api-tls\") pod \"713ca469-cb40-47b0-a3ce-02c2c7bdf021\" (UID: \"713ca469-cb40-47b0-a3ce-02c2c7bdf021\") " Jul 21 13:27:11.812085 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:11.811971 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/713ca469-cb40-47b0-a3ce-02c2c7bdf021-openshift-service-ca\") pod \"713ca469-cb40-47b0-a3ce-02c2c7bdf021\" (UID: \"713ca469-cb40-47b0-a3ce-02c2c7bdf021\") " Jul 21 13:27:11.812355 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:11.812334 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/713ca469-cb40-47b0-a3ce-02c2c7bdf021-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "713ca469-cb40-47b0-a3ce-02c2c7bdf021" (UID: "713ca469-cb40-47b0-a3ce-02c2c7bdf021"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:27:11.814190 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:11.814157 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/713ca469-cb40-47b0-a3ce-02c2c7bdf021-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "713ca469-cb40-47b0-a3ce-02c2c7bdf021" (UID: "713ca469-cb40-47b0-a3ce-02c2c7bdf021"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 13:27:11.814190 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:11.814172 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/713ca469-cb40-47b0-a3ce-02c2c7bdf021-kube-api-access-5dwl7" (OuterVolumeSpecName: "kube-api-access-5dwl7") pod "713ca469-cb40-47b0-a3ce-02c2c7bdf021" (UID: "713ca469-cb40-47b0-a3ce-02c2c7bdf021"). InnerVolumeSpecName "kube-api-access-5dwl7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:27:11.913542 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:11.913456 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/713ca469-cb40-47b0-a3ce-02c2c7bdf021-maas-api-tls\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:27:11.913542 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:11.913486 2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/713ca469-cb40-47b0-a3ce-02c2c7bdf021-openshift-service-ca\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:27:11.913542 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:11.913500 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dwl7\" (UniqueName: \"kubernetes.io/projected/713ca469-cb40-47b0-a3ce-02c2c7bdf021-kube-api-access-5dwl7\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:27:12.087026 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:12.086994 2555 generic.go:356] "Generic (PLEG): container finished" podID="713ca469-cb40-47b0-a3ce-02c2c7bdf021" containerID="eb401e20c8c312545721408a4de3d21127b6849a75b3e76b89b101920bbd5676" exitCode=0 Jul 21 13:27:12.087207 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:12.087045 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" Jul 21 13:27:12.087207 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:12.087087 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" event={"ID":"713ca469-cb40-47b0-a3ce-02c2c7bdf021","Type":"ContainerDied","Data":"eb401e20c8c312545721408a4de3d21127b6849a75b3e76b89b101920bbd5676"} Jul 21 13:27:12.087207 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:12.087144 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2" event={"ID":"713ca469-cb40-47b0-a3ce-02c2c7bdf021","Type":"ContainerDied","Data":"5b70ef867c6fc136136980833ec4c7d7f70a84868e59bf2549fc07c74854062a"} Jul 21 13:27:12.087207 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:12.087172 2555 scope.go:117] "RemoveContainer" containerID="eb401e20c8c312545721408a4de3d21127b6849a75b3e76b89b101920bbd5676" Jul 21 13:27:12.096479 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:12.096457 2555 scope.go:117] "RemoveContainer" containerID="eb401e20c8c312545721408a4de3d21127b6849a75b3e76b89b101920bbd5676" Jul 21 13:27:12.096870 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:27:12.096849 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb401e20c8c312545721408a4de3d21127b6849a75b3e76b89b101920bbd5676\": container with ID starting with eb401e20c8c312545721408a4de3d21127b6849a75b3e76b89b101920bbd5676 not found: ID does not exist" containerID="eb401e20c8c312545721408a4de3d21127b6849a75b3e76b89b101920bbd5676" Jul 21 13:27:12.096955 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:12.096879 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb401e20c8c312545721408a4de3d21127b6849a75b3e76b89b101920bbd5676"} err="failed to get container status \"eb401e20c8c312545721408a4de3d21127b6849a75b3e76b89b101920bbd5676\": rpc error: code = NotFound desc = could not find container \"eb401e20c8c312545721408a4de3d21127b6849a75b3e76b89b101920bbd5676\": container with ID starting with eb401e20c8c312545721408a4de3d21127b6849a75b3e76b89b101920bbd5676 not found: ID does not exist" Jul 21 13:27:12.109101 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:12.109068 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2"] Jul 21 13:27:12.120782 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:12.118440 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-cfed01-6b7b7cf46c-gfsk2"] Jul 21 13:27:13.925985 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:13.925947 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="713ca469-cb40-47b0-a3ce-02c2c7bdf021" path="/var/lib/kubelet/pods/713ca469-cb40-47b0-a3ce-02c2c7bdf021/volumes" Jul 21 13:27:23.100288 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:23.100249 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt"] Jul 21 13:27:23.100824 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:23.100498 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" podUID="06adbbc7-a40e-48c5-bd70-a93ec25e61ee" containerName="istio-proxy" containerID="cri-o://e0c86f64c65cfa304facb53b973bb0ba27b2e6147ffbc8eab7794c6c65d1b9a7" gracePeriod=30 Jul 21 13:27:24.556622 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:24.556557 2555 patch_prober.go:28] interesting pod/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 21 13:27:24.557151 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:24.556654 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" podUID="06adbbc7-a40e-48c5-bd70-a93ec25e61ee" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 21 13:27:28.368691 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.368616 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:27:28.471897 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.471813 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jldjq\" (UniqueName: \"kubernetes.io/projected/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-kube-api-access-jldjq\") pod \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " Jul 21 13:27:28.471897 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.471887 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-workload-certs\") pod \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " Jul 21 13:27:28.472140 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.471925 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-envoy\") pod \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " Jul 21 13:27:28.472140 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.471961 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-credential-socket\") pod \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " Jul 21 13:27:28.472140 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.472002 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-token\") pod \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " Jul 21 13:27:28.472140 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.472025 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-workload-socket\") pod \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " Jul 21 13:27:28.472140 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.472099 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-data\") pod \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " Jul 21 13:27:28.472140 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.472137 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istiod-ca-cert\") pod \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " Jul 21 13:27:28.472451 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.472163 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-podinfo\") pod \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\" (UID: \"06adbbc7-a40e-48c5-bd70-a93ec25e61ee\") " Jul 21 13:27:28.472451 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.472245 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "06adbbc7-a40e-48c5-bd70-a93ec25e61ee" (UID: "06adbbc7-a40e-48c5-bd70-a93ec25e61ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:27:28.472451 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.472326 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "06adbbc7-a40e-48c5-bd70-a93ec25e61ee" (UID: "06adbbc7-a40e-48c5-bd70-a93ec25e61ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:27:28.472451 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.472344 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "06adbbc7-a40e-48c5-bd70-a93ec25e61ee" (UID: "06adbbc7-a40e-48c5-bd70-a93ec25e61ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:27:28.472708 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.472520 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-workload-certs\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:27:28.472708 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.472539 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-credential-socket\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:27:28.472708 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.472554 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-workload-socket\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:27:28.472708 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.472671 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "06adbbc7-a40e-48c5-bd70-a93ec25e61ee" (UID: "06adbbc7-a40e-48c5-bd70-a93ec25e61ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 13:27:28.472987 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.472968 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-data" (OuterVolumeSpecName: "istio-data") pod "06adbbc7-a40e-48c5-bd70-a93ec25e61ee" (UID: "06adbbc7-a40e-48c5-bd70-a93ec25e61ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:27:28.474906 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.474868 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "06adbbc7-a40e-48c5-bd70-a93ec25e61ee" (UID: "06adbbc7-a40e-48c5-bd70-a93ec25e61ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 13:27:28.475028 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.474926 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-token" (OuterVolumeSpecName: "istio-token") pod "06adbbc7-a40e-48c5-bd70-a93ec25e61ee" (UID: "06adbbc7-a40e-48c5-bd70-a93ec25e61ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:27:28.475028 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.474938 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-kube-api-access-jldjq" (OuterVolumeSpecName: "kube-api-access-jldjq") pod "06adbbc7-a40e-48c5-bd70-a93ec25e61ee" (UID: "06adbbc7-a40e-48c5-bd70-a93ec25e61ee"). InnerVolumeSpecName "kube-api-access-jldjq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 13:27:28.475028 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.474961 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "06adbbc7-a40e-48c5-bd70-a93ec25e61ee" (UID: "06adbbc7-a40e-48c5-bd70-a93ec25e61ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 13:27:28.573659 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.573621 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-token\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:27:28.573659 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.573654 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-data\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:27:28.573659 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.573663 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istiod-ca-cert\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:27:28.573659 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.573670 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-podinfo\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:27:28.573903 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.573678 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jldjq\" (UniqueName: \"kubernetes.io/projected/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-kube-api-access-jldjq\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:27:28.573903 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:28.573687 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06adbbc7-a40e-48c5-bd70-a93ec25e61ee-istio-envoy\") on node \"ip-10-0-133-39.ec2.internal\" DevicePath \"\"" Jul 21 13:27:29.182718 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:29.182686 2555 generic.go:356] "Generic (PLEG): container finished" podID="06adbbc7-a40e-48c5-bd70-a93ec25e61ee" containerID="e0c86f64c65cfa304facb53b973bb0ba27b2e6147ffbc8eab7794c6c65d1b9a7" exitCode=0 Jul 21 13:27:29.182912 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:29.182739 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" Jul 21 13:27:29.182912 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:29.182777 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" event={"ID":"06adbbc7-a40e-48c5-bd70-a93ec25e61ee","Type":"ContainerDied","Data":"e0c86f64c65cfa304facb53b973bb0ba27b2e6147ffbc8eab7794c6c65d1b9a7"} Jul 21 13:27:29.182912 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:29.182839 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt" event={"ID":"06adbbc7-a40e-48c5-bd70-a93ec25e61ee","Type":"ContainerDied","Data":"0ea9cd200aacd20b622fe9c70468aa5494a02885e8f300027e0f0f789a974cf8"} Jul 21 13:27:29.182912 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:29.182871 2555 scope.go:117] "RemoveContainer" containerID="e0c86f64c65cfa304facb53b973bb0ba27b2e6147ffbc8eab7794c6c65d1b9a7" Jul 21 13:27:29.192792 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:29.192774 2555 scope.go:117] "RemoveContainer" containerID="b34f1fb52c37c82deaa5b51aa6fa588da43e4a3a548c649a56c54893b6c86a55" Jul 21 13:27:29.202139 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:29.202119 2555 scope.go:117] "RemoveContainer" containerID="e0c86f64c65cfa304facb53b973bb0ba27b2e6147ffbc8eab7794c6c65d1b9a7" Jul 21 13:27:29.202439 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:27:29.202419 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0c86f64c65cfa304facb53b973bb0ba27b2e6147ffbc8eab7794c6c65d1b9a7\": container with ID starting with e0c86f64c65cfa304facb53b973bb0ba27b2e6147ffbc8eab7794c6c65d1b9a7 not found: ID does not exist" containerID="e0c86f64c65cfa304facb53b973bb0ba27b2e6147ffbc8eab7794c6c65d1b9a7" Jul 21 13:27:29.202488 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:29.202449 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0c86f64c65cfa304facb53b973bb0ba27b2e6147ffbc8eab7794c6c65d1b9a7"} err="failed to get container status \"e0c86f64c65cfa304facb53b973bb0ba27b2e6147ffbc8eab7794c6c65d1b9a7\": rpc error: code = NotFound desc = could not find container \"e0c86f64c65cfa304facb53b973bb0ba27b2e6147ffbc8eab7794c6c65d1b9a7\": container with ID starting with e0c86f64c65cfa304facb53b973bb0ba27b2e6147ffbc8eab7794c6c65d1b9a7 not found: ID does not exist" Jul 21 13:27:29.202488 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:29.202468 2555 scope.go:117] "RemoveContainer" containerID="b34f1fb52c37c82deaa5b51aa6fa588da43e4a3a548c649a56c54893b6c86a55" Jul 21 13:27:29.202748 ip-10-0-133-39 kubenswrapper[2555]: E0721 13:27:29.202729 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b34f1fb52c37c82deaa5b51aa6fa588da43e4a3a548c649a56c54893b6c86a55\": container with ID starting with b34f1fb52c37c82deaa5b51aa6fa588da43e4a3a548c649a56c54893b6c86a55 not found: ID does not exist" containerID="b34f1fb52c37c82deaa5b51aa6fa588da43e4a3a548c649a56c54893b6c86a55" Jul 21 13:27:29.202818 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:29.202757 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b34f1fb52c37c82deaa5b51aa6fa588da43e4a3a548c649a56c54893b6c86a55"} err="failed to get container status \"b34f1fb52c37c82deaa5b51aa6fa588da43e4a3a548c649a56c54893b6c86a55\": rpc error: code = NotFound desc = could not find container \"b34f1fb52c37c82deaa5b51aa6fa588da43e4a3a548c649a56c54893b6c86a55\": container with ID starting with b34f1fb52c37c82deaa5b51aa6fa588da43e4a3a548c649a56c54893b6c86a55 not found: ID does not exist" Jul 21 13:27:29.206654 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:29.206628 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt"] Jul 21 13:27:29.213399 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:29.213374 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-cfed01-openshift-default-5499bd9bd7-nspbt"] Jul 21 13:27:29.823791 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:29.823763 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6bf95d5449-79kjl_bdcd5f09-697c-43de-bdc5-da5cebcbcae5/manager/0.log" Jul 21 13:27:29.926712 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:29.926680 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06adbbc7-a40e-48c5-bd70-a93ec25e61ee" path="/var/lib/kubelet/pods/06adbbc7-a40e-48c5-bd70-a93ec25e61ee/volumes" Jul 21 13:27:30.347269 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:30.347241 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-579d89bddd-pq59b_82740182-3d7b-4d3c-8d49-8ed65c1ab143/copy-manifests/0.log" Jul 21 13:27:30.362696 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:30.362665 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-579d89bddd-pq59b_82740182-3d7b-4d3c-8d49-8ed65c1ab143/manager/0.log" Jul 21 13:27:32.059126 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:32.059087 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr_60005c66-98c1-4cc0-8018-20461298604e/pull/0.log" Jul 21 13:27:32.067414 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:32.067388 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr_60005c66-98c1-4cc0-8018-20461298604e/extract/0.log" Jul 21 13:27:32.075250 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:32.075222 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr_60005c66-98c1-4cc0-8018-20461298604e/util/0.log" Jul 21 13:27:32.214251 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:32.214198 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf_952c588a-0e50-48c7-8460-b5bfba7189d8/util/0.log" Jul 21 13:27:32.222148 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:32.222119 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf_952c588a-0e50-48c7-8460-b5bfba7189d8/pull/0.log" Jul 21 13:27:32.229842 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:32.229808 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf_952c588a-0e50-48c7-8460-b5bfba7189d8/extract/0.log" Jul 21 13:27:32.354301 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:32.354268 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk_3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf/util/0.log" Jul 21 13:27:32.362912 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:32.362886 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk_3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf/pull/0.log" Jul 21 13:27:32.371721 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:32.371698 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk_3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf/extract/0.log" Jul 21 13:27:32.495236 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:32.495198 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl_caf82597-ec5e-4e4c-8546-2123c195bcbd/util/0.log" Jul 21 13:27:32.506915 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:32.506888 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl_caf82597-ec5e-4e4c-8546-2123c195bcbd/pull/0.log" Jul 21 13:27:32.514863 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:32.514843 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl_caf82597-ec5e-4e4c-8546-2123c195bcbd/extract/0.log" Jul 21 13:27:32.901610 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:32.901565 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-g5hcm_b481abe3-4169-4a16-9449-89ae4fcb3ba9/manager/0.log" Jul 21 13:27:33.298413 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:33.298329 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-pf6gh_afd6f2df-034b-45df-811f-0d6c7ec23ac3/manager/0.log" Jul 21 13:27:34.051589 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:34.051557 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-wqhww_486642b7-385b-496b-9ee8-f924d76c0ba6/discovery/0.log" Jul 21 13:27:34.800697 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:34.800670 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6974856856-hqzjg_9bb8c882-1f66-4054-8f88-80a3fd392806/router/0.log" Jul 21 13:27:35.429231 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:35.429202 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-rjvxh_84bd219d-205e-4c6c-8382-f007b8fc54da/main/0.log" Jul 21 13:27:35.552772 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:35.552742 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f2rd2k_47702c39-f4bf-4057-963b-dadf58b60cd8/main/0.log" Jul 21 13:27:45.146556 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:45.146505 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vtvm8_a807b465-26c5-46ad-8abb-09a4f7716bee/global-pull-secret-syncer/0.log" Jul 21 13:27:45.346487 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:45.346453 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-x72xs_aaa90a3d-1518-43bb-bff2-319c194a66ea/konnectivity-agent/0.log" Jul 21 13:27:45.486246 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:45.486171 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-39.ec2.internal_aa32358a6864982ffe270d580961f111/haproxy/0.log" Jul 21 13:27:49.748924 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:49.748883 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr_60005c66-98c1-4cc0-8018-20461298604e/extract/0.log" Jul 21 13:27:49.783437 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:49.783401 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr_60005c66-98c1-4cc0-8018-20461298604e/util/0.log" Jul 21 13:27:49.815481 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:49.815438 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gs8nr_60005c66-98c1-4cc0-8018-20461298604e/pull/0.log" Jul 21 13:27:49.852487 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:49.852444 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf_952c588a-0e50-48c7-8460-b5bfba7189d8/extract/0.log" Jul 21 13:27:49.883353 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:49.883328 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf_952c588a-0e50-48c7-8460-b5bfba7189d8/util/0.log" Jul 21 13:27:49.917012 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:49.916979 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v8xzf_952c588a-0e50-48c7-8460-b5bfba7189d8/pull/0.log" Jul 21 13:27:49.954356 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:49.954307 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk_3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf/extract/0.log" Jul 21 13:27:50.000979 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:50.000881 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk_3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf/util/0.log" Jul 21 13:27:50.040754 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:50.040700 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73r9shk_3cbaf6ab-5fa5-416a-ba27-fc99281d2aaf/pull/0.log" Jul 21 13:27:50.078188 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:50.078122 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl_caf82597-ec5e-4e4c-8546-2123c195bcbd/extract/0.log" Jul 21 13:27:50.109188 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:50.109160 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl_caf82597-ec5e-4e4c-8546-2123c195bcbd/util/0.log" Jul 21 13:27:50.137502 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:50.137480 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssbgl_caf82597-ec5e-4e4c-8546-2123c195bcbd/pull/0.log" Jul 21 13:27:50.586993 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:50.586961 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-g5hcm_b481abe3-4169-4a16-9449-89ae4fcb3ba9/manager/0.log" Jul 21 13:27:51.011806 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:51.011703 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-pf6gh_afd6f2df-034b-45df-811f-0d6c7ec23ac3/manager/0.log" Jul 21 13:27:53.481570 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:53.481541 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d4t5z_0dd584cd-f447-4f04-bc66-efb40e36578f/node-exporter/0.log" Jul 21 13:27:53.512239 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:53.512211 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d4t5z_0dd584cd-f447-4f04-bc66-efb40e36578f/kube-rbac-proxy/0.log" Jul 21 13:27:53.540762 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:53.540740 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d4t5z_0dd584cd-f447-4f04-bc66-efb40e36578f/init-textfile/0.log" Jul 21 13:27:53.673903 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:53.673875 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-l72lr_631f4a0a-8dab-49f8-a4da-b493b6d1343a/kube-rbac-proxy-main/0.log" Jul 21 13:27:53.702567 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:53.702547 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-l72lr_631f4a0a-8dab-49f8-a4da-b493b6d1343a/kube-rbac-proxy-self/0.log" Jul 21 13:27:53.730737 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:53.730717 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-l72lr_631f4a0a-8dab-49f8-a4da-b493b6d1343a/openshift-state-metrics/0.log" Jul 21 13:27:54.044050 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.044020 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-lhv2d_c4a34804-6773-4cb3-8abe-8e1a965e23e2/prometheus-operator/0.log" Jul 21 13:27:54.079798 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.079764 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-lhv2d_c4a34804-6773-4cb3-8abe-8e1a965e23e2/kube-rbac-proxy/0.log" Jul 21 13:27:54.120855 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.120829 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-72bwk_e1e07274-8366-4ce0-83f1-9994b75b963b/prometheus-operator-admission-webhook/0.log" Jul 21 13:27:54.340991 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.340965 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f8d7495db-slrlf_b4ad0e99-86fb-4495-9ed2-a977b6b96afc/thanos-query/0.log" Jul 21 13:27:54.384030 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.384002 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f8d7495db-slrlf_b4ad0e99-86fb-4495-9ed2-a977b6b96afc/kube-rbac-proxy-web/0.log" Jul 21 13:27:54.415833 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.415811 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f8d7495db-slrlf_b4ad0e99-86fb-4495-9ed2-a977b6b96afc/kube-rbac-proxy/0.log" Jul 21 13:27:54.446473 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.446445 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f8d7495db-slrlf_b4ad0e99-86fb-4495-9ed2-a977b6b96afc/prom-label-proxy/0.log" Jul 21 13:27:54.478653 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.478625 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f8d7495db-slrlf_b4ad0e99-86fb-4495-9ed2-a977b6b96afc/kube-rbac-proxy-rules/0.log" Jul 21 13:27:54.509098 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.509068 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f8d7495db-slrlf_b4ad0e99-86fb-4495-9ed2-a977b6b96afc/kube-rbac-proxy-metrics/0.log" Jul 21 13:27:54.774639 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.774554 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px"] Jul 21 13:27:54.774956 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.774943 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06adbbc7-a40e-48c5-bd70-a93ec25e61ee" containerName="istio-proxy" Jul 21 13:27:54.774998 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.774959 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="06adbbc7-a40e-48c5-bd70-a93ec25e61ee" containerName="istio-proxy" Jul 21 13:27:54.774998 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.774976 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7091c316-c17d-4263-99aa-8f4741dcf257" containerName="maas-api" Jul 21 13:27:54.774998 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.774982 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="7091c316-c17d-4263-99aa-8f4741dcf257" containerName="maas-api" Jul 21 13:27:54.774998 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.774992 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="713ca469-cb40-47b0-a3ce-02c2c7bdf021" containerName="maas-api" Jul 21 13:27:54.775126 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.774999 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="713ca469-cb40-47b0-a3ce-02c2c7bdf021" containerName="maas-api" Jul 21 13:27:54.775126 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.775059 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="06adbbc7-a40e-48c5-bd70-a93ec25e61ee" containerName="istio-proxy" Jul 21 13:27:54.775126 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.775069 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="713ca469-cb40-47b0-a3ce-02c2c7bdf021" containerName="maas-api" Jul 21 13:27:54.775126 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.775077 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="7091c316-c17d-4263-99aa-8f4741dcf257" containerName="maas-api" Jul 21 13:27:54.778298 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.778281 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" Jul 21 13:27:54.780653 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.780628 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s9mdb\"/\"openshift-service-ca.crt\"" Jul 21 13:27:54.780861 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.780762 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-s9mdb\"/\"default-dockercfg-kpxh2\"" Jul 21 13:27:54.781123 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.781109 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s9mdb\"/\"kube-root-ca.crt\"" Jul 21 13:27:54.791036 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.791009 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px"] Jul 21 13:27:54.881713 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.881682 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/40c2465e-4524-41c6-8186-8f5cd688478c-proc\") pod \"perf-node-gather-daemonset-z55px\" (UID: \"40c2465e-4524-41c6-8186-8f5cd688478c\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" Jul 21 13:27:54.881895 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.881720 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/40c2465e-4524-41c6-8186-8f5cd688478c-lib-modules\") pod \"perf-node-gather-daemonset-z55px\" (UID: \"40c2465e-4524-41c6-8186-8f5cd688478c\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" Jul 21 13:27:54.881895 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.881803 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65d4x\" (UniqueName: \"kubernetes.io/projected/40c2465e-4524-41c6-8186-8f5cd688478c-kube-api-access-65d4x\") pod \"perf-node-gather-daemonset-z55px\" (UID: \"40c2465e-4524-41c6-8186-8f5cd688478c\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" Jul 21 13:27:54.881895 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.881836 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40c2465e-4524-41c6-8186-8f5cd688478c-sys\") pod \"perf-node-gather-daemonset-z55px\" (UID: \"40c2465e-4524-41c6-8186-8f5cd688478c\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" Jul 21 13:27:54.881895 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.881856 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/40c2465e-4524-41c6-8186-8f5cd688478c-podres\") pod \"perf-node-gather-daemonset-z55px\" (UID: \"40c2465e-4524-41c6-8186-8f5cd688478c\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" Jul 21 13:27:54.983052 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.983014 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65d4x\" (UniqueName: \"kubernetes.io/projected/40c2465e-4524-41c6-8186-8f5cd688478c-kube-api-access-65d4x\") pod \"perf-node-gather-daemonset-z55px\" (UID: \"40c2465e-4524-41c6-8186-8f5cd688478c\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" Jul 21 13:27:54.983298 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.983083 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40c2465e-4524-41c6-8186-8f5cd688478c-sys\") pod \"perf-node-gather-daemonset-z55px\" (UID: \"40c2465e-4524-41c6-8186-8f5cd688478c\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" Jul 21 13:27:54.983298 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.983140 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/40c2465e-4524-41c6-8186-8f5cd688478c-podres\") pod \"perf-node-gather-daemonset-z55px\" (UID: \"40c2465e-4524-41c6-8186-8f5cd688478c\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" Jul 21 13:27:54.983298 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.983199 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/40c2465e-4524-41c6-8186-8f5cd688478c-proc\") pod \"perf-node-gather-daemonset-z55px\" (UID: \"40c2465e-4524-41c6-8186-8f5cd688478c\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" Jul 21 13:27:54.983298 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.983257 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/40c2465e-4524-41c6-8186-8f5cd688478c-lib-modules\") pod \"perf-node-gather-daemonset-z55px\" (UID: \"40c2465e-4524-41c6-8186-8f5cd688478c\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" Jul 21 13:27:54.983554 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.983345 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40c2465e-4524-41c6-8186-8f5cd688478c-sys\") pod \"perf-node-gather-daemonset-z55px\" (UID: \"40c2465e-4524-41c6-8186-8f5cd688478c\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" Jul 21 13:27:54.983554 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.983353 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/40c2465e-4524-41c6-8186-8f5cd688478c-podres\") pod \"perf-node-gather-daemonset-z55px\" (UID: \"40c2465e-4524-41c6-8186-8f5cd688478c\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" Jul 21 13:27:54.983554 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.983420 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/40c2465e-4524-41c6-8186-8f5cd688478c-lib-modules\") pod \"perf-node-gather-daemonset-z55px\" (UID: \"40c2465e-4524-41c6-8186-8f5cd688478c\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" Jul 21 13:27:54.983554 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.983490 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/40c2465e-4524-41c6-8186-8f5cd688478c-proc\") pod \"perf-node-gather-daemonset-z55px\" (UID: \"40c2465e-4524-41c6-8186-8f5cd688478c\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" Jul 21 13:27:54.994720 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:54.994696 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65d4x\" (UniqueName: \"kubernetes.io/projected/40c2465e-4524-41c6-8186-8f5cd688478c-kube-api-access-65d4x\") pod \"perf-node-gather-daemonset-z55px\" (UID: \"40c2465e-4524-41c6-8186-8f5cd688478c\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" Jul 21 13:27:55.089221 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:55.089192 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" Jul 21 13:27:55.232870 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:55.232687 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px"] Jul 21 13:27:55.235130 ip-10-0-133-39 kubenswrapper[2555]: W0721 13:27:55.235104 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod40c2465e_4524_41c6_8186_8f5cd688478c.slice/crio-a1362fc7c58e71b7dcf9c78e4b344c109dd54733928d957a1f1d7b6d14d8ed1f WatchSource:0}: Error finding container a1362fc7c58e71b7dcf9c78e4b344c109dd54733928d957a1f1d7b6d14d8ed1f: Status 404 returned error can't find the container with id a1362fc7c58e71b7dcf9c78e4b344c109dd54733928d957a1f1d7b6d14d8ed1f Jul 21 13:27:55.323548 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:55.323515 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" event={"ID":"40c2465e-4524-41c6-8186-8f5cd688478c","Type":"ContainerStarted","Data":"a1362fc7c58e71b7dcf9c78e4b344c109dd54733928d957a1f1d7b6d14d8ed1f"} Jul 21 13:27:55.827954 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:55.827928 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-tcq6l_96c83e06-37e1-4bb8-82ab-3db5462a7528/networking-console-plugin/0.log" Jul 21 13:27:56.330089 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:56.330045 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" event={"ID":"40c2465e-4524-41c6-8186-8f5cd688478c","Type":"ContainerStarted","Data":"4c1e8c0a24aed92d764063a290a12e2a76d0b4667c57dfa2c16fac2f5c903248"} Jul 21 13:27:56.330272 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:56.330130 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" Jul 21 13:27:56.355823 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:56.355771 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" podStartSLOduration=2.355752986 podStartE2EDuration="2.355752986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:27:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:27:56.355744502 +0000 UTC m=+2898.974037454" watchObservedRunningTime="2026-07-21 13:27:56.355752986 +0000 UTC m=+2898.974045934" Jul 21 13:27:56.425481 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:56.425446 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/2.log" Jul 21 13:27:56.429464 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:56.429440 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qnc6t_af138219-695f-4f5d-8a2f-df2c0a89f117/console-operator/3.log" Jul 21 13:27:57.021081 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:57.021042 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-zlwln_73508863-33d6-4f43-b1eb-437b303a601c/download-server/0.log" Jul 21 13:27:59.150655 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:59.150623 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gcm9k_de273e85-f101-43f9-96ec-b7102d29ac49/dns/0.log" Jul 21 13:27:59.196248 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:59.196217 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gcm9k_de273e85-f101-43f9-96ec-b7102d29ac49/kube-rbac-proxy/0.log" Jul 21 13:27:59.292994 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:27:59.292966 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-chngz_58089888-e584-4b42-ae32-e8074321df87/dns-node-resolver/0.log" Jul 21 13:28:00.131716 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:28:00.131682 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v98xb_ba7b78ab-59ae-485e-93cc-8005bd5b9a69/node-ca/0.log" Jul 21 13:28:01.267264 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:28:01.267231 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-wqhww_486642b7-385b-496b-9ee8-f924d76c0ba6/discovery/0.log" Jul 21 13:28:01.489144 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:28:01.489118 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6974856856-hqzjg_9bb8c882-1f66-4054-8f88-80a3fd392806/router/0.log" Jul 21 13:28:02.345717 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:28:02.345680 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-z55px" Jul 21 13:28:02.349708 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:28:02.349687 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f6p9g_02d7c116-6592-44ac-89f4-8cba0c30c18b/serve-healthcheck-canary/0.log" Jul 21 13:28:03.072735 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:28:03.072707 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v562r_037a9656-a505-40ec-b7f8-03b73974213a/kube-rbac-proxy/0.log" Jul 21 13:28:03.098520 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:28:03.098492 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v562r_037a9656-a505-40ec-b7f8-03b73974213a/exporter/0.log" Jul 21 13:28:03.126503 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:28:03.126477 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v562r_037a9656-a505-40ec-b7f8-03b73974213a/extractor/0.log" Jul 21 13:28:05.482152 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:28:05.482118 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-wqhww_486642b7-385b-496b-9ee8-f924d76c0ba6/discovery/0.log" Jul 21 13:28:05.649964 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:28:05.649932 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6974856856-hqzjg_9bb8c882-1f66-4054-8f88-80a3fd392806/router/0.log" Jul 21 13:28:06.006533 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:28:06.006498 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6bf95d5449-79kjl_bdcd5f09-697c-43de-bdc5-da5cebcbcae5/manager/0.log" Jul 21 13:28:06.292001 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:28:06.291965 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-579d89bddd-pq59b_82740182-3d7b-4d3c-8d49-8ed65c1ab143/manager/0.log" Jul 21 13:28:06.314151 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:28:06.314124 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-579d89bddd-pq59b_82740182-3d7b-4d3c-8d49-8ed65c1ab143/copy-manifests/0.log" Jul 21 13:28:07.087112 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:28:07.087021 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-wqhww_486642b7-385b-496b-9ee8-f924d76c0ba6/discovery/0.log" Jul 21 13:28:07.247511 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:28:07.247484 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6974856856-hqzjg_9bb8c882-1f66-4054-8f88-80a3fd392806/router/0.log" Jul 21 13:28:07.606828 ip-10-0-133-39 kubenswrapper[2555]: I0721 13:28:07.606797 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-84bfb648b5-sfv9t_590aa5c3-3489-4866-ace2-1cb44b5d3851/manager/0.log"