Jul 27 20:49:19.473164 ip-10-0-131-232 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 27 20:49:19.473198 ip-10-0-131-232 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 27 20:49:19.473206 ip-10-0-131-232 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 27 20:49:19.473435 ip-10-0-131-232 systemd[1]: Failed to start Kubernetes Kubelet. Jul 27 20:49:29.722560 ip-10-0-131-232 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 27 20:49:29.722582 ip-10-0-131-232 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 9be49b2c12cc4e70a9710aea502a9c2f -- Jul 27 20:51:10.018706 ip-10-0-131-232 systemd[1]: Starting Kubernetes Kubelet... Jul 27 20:51:10.469563 ip-10-0-131-232 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 27 20:51:10.469563 ip-10-0-131-232 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 27 20:51:10.469563 ip-10-0-131-232 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 27 20:51:10.469563 ip-10-0-131-232 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 27 20:51:10.469563 ip-10-0-131-232 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 27 20:51:10.471267 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.471154 2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 27 20:51:10.477370 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477354 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 20:51:10.477370 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477370 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 20:51:10.477455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477374 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 20:51:10.477455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477378 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 20:51:10.477455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477381 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 20:51:10.477455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477386 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 20:51:10.477455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477390 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 20:51:10.477455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477393 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 20:51:10.477455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477401 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 20:51:10.477455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477404 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 20:51:10.477455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477407 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 20:51:10.477455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477410 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 20:51:10.477455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477413 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 20:51:10.477455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477415 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 20:51:10.477455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477418 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 20:51:10.477455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477421 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 20:51:10.477455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477423 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 20:51:10.477455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477437 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 20:51:10.477455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477440 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 20:51:10.477455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477443 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 20:51:10.477455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477446 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477448 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477451 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477453 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477456 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477458 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477461 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477464 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477467 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477470 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477472 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477475 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477477 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477480 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477482 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477485 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477487 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477490 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477493 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477495 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 20:51:10.477936 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477498 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 20:51:10.478772 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477500 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 20:51:10.478772 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477502 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 20:51:10.478772 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477505 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 20:51:10.478772 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477507 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 20:51:10.478772 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477510 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 20:51:10.478772 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477512 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 20:51:10.478772 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477515 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 20:51:10.478772 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477517 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 20:51:10.478772 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477520 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 20:51:10.478772 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477522 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 20:51:10.478772 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477526 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 20:51:10.478772 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477530 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 20:51:10.478772 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477533 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 20:51:10.478772 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477536 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 20:51:10.478772 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477539 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 20:51:10.478772 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477543 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 20:51:10.478772 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477547 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 20:51:10.478772 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477550 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 20:51:10.478772 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477552 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477556 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477559 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477562 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477565 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477567 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477569 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477572 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477574 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477577 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477579 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477582 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477584 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477587 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477589 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477591 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477594 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477596 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477598 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477601 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 20:51:10.479373 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477606 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 20:51:10.479904 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477609 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 20:51:10.479904 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477611 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 20:51:10.479904 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477614 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 20:51:10.479904 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477616 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 20:51:10.479904 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477619 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 20:51:10.479904 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477622 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 20:51:10.479904 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477625 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 20:51:10.479904 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477628 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 20:51:10.479904 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477630 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 20:51:10.479904 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477633 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 20:51:10.479904 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477636 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 20:51:10.479904 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477638 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 20:51:10.479904 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477641 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 20:51:10.479904 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477644 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 20:51:10.479904 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477646 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 20:51:10.479904 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477649 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 20:51:10.479904 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477651 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 20:51:10.479904 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477654 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 20:51:10.479904 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.477656 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 20:51:10.480585 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480569 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 20:51:10.480585 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480583 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480587 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480591 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480594 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480597 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480600 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480603 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480605 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480609 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480612 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480615 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480617 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480620 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480623 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480625 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480628 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480631 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480635 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480639 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480642 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 20:51:10.480647 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480645 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 20:51:10.481132 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480648 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 20:51:10.481132 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480651 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 20:51:10.481132 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480654 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 20:51:10.481132 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480656 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 20:51:10.481132 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480659 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 20:51:10.481132 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480663 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 20:51:10.481132 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480666 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 20:51:10.481132 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480669 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 20:51:10.481132 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480672 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 20:51:10.481132 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480675 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 20:51:10.481132 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480677 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 20:51:10.481132 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480680 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 20:51:10.481132 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480682 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 20:51:10.481132 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480685 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 20:51:10.481132 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480688 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 20:51:10.481132 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480691 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 20:51:10.481132 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480693 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 27 20:51:10.481132 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480696 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 20:51:10.481132 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480698 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480700 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480703 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480705 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480707 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480710 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480712 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480715 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480717 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480719 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480722 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480724 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480728 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480731 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480735 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480738 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480741 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480744 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480746 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480749 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 20:51:10.481617 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480751 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480753 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480756 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480758 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480761 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480763 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480766 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480768 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480772 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480774 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480776 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480779 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480781 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480784 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480786 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480788 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480791 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480793 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480795 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480798 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 20:51:10.482105 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480800 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480803 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480805 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480808 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480811 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480813 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480816 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480819 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480821 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480824 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480826 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480829 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480832 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480834 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480837 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480839 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480842 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480844 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.480847 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481256 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 20:51:10.482580 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481261 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481265 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481268 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481271 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481273 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481276 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481278 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481281 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481283 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481286 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481288 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481290 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481293 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481295 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481298 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481301 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481304 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481306 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481309 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481312 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 20:51:10.483050 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481314 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481317 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481319 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481322 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481324 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481327 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481337 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481341 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481344 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481346 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481349 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481352 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481354 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481357 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481359 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481361 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481364 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481366 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481369 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481371 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 20:51:10.483541 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481374 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481377 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481379 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481381 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481384 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481386 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481389 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481392 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481395 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481398 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481401 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481404 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481406 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481408 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481411 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481414 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481416 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481419 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481422 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481424 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 20:51:10.484026 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481442 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 20:51:10.484545 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481445 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 20:51:10.484545 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481447 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 20:51:10.484545 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481450 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 20:51:10.484545 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481453 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 20:51:10.484545 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481455 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 20:51:10.484545 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481458 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 20:51:10.484545 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481462 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 20:51:10.484545 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481466 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 20:51:10.484545 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481469 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 20:51:10.484545 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481473 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 20:51:10.484545 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481477 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 20:51:10.484545 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481479 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 20:51:10.484545 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481482 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 20:51:10.484545 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481487 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 20:51:10.484545 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481490 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 20:51:10.484545 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481493 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 20:51:10.484545 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481496 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 20:51:10.484545 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481498 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 20:51:10.484545 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481501 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481503 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481506 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481510 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481512 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481515 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481517 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481520 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481522 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481525 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481528 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481530 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481532 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481535 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481537 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481540 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481542 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481545 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.481547 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481621 2565 flags.go:64] FLAG: --address="0.0.0.0" Jul 27 20:51:10.484996 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481628 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481634 2565 flags.go:64] FLAG: --anonymous-auth="true" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481639 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481642 2565 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481645 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481649 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481653 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481656 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481659 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481663 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481666 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481669 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481672 2565 flags.go:64] FLAG: --cgroup-root="" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481675 2565 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481678 2565 flags.go:64] FLAG: --client-ca-file="" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481681 2565 flags.go:64] FLAG: --cloud-provider="external" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481685 2565 flags.go:64] FLAG: --cluster-dns="[]" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481690 2565 flags.go:64] FLAG: --cluster-domain="" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481692 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481695 2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481698 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481701 2565 flags.go:64] FLAG: --container-log-max-files="5" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481705 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 27 20:51:10.485586 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481708 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481710 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481714 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481717 2565 flags.go:64] FLAG: --contention-profiling="false" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481719 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481722 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481725 2565 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481728 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481732 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481735 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481738 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481740 2565 flags.go:64] FLAG: --enable-load-reader="false" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481743 2565 flags.go:64] FLAG: --enable-server="true" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481746 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481750 2565 flags.go:64] FLAG: --event-burst="100" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481753 2565 flags.go:64] FLAG: --event-qps="50" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481756 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481759 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481761 2565 flags.go:64] FLAG: --eviction-hard="" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481765 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481767 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481770 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481773 2565 flags.go:64] FLAG: --eviction-soft="" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481777 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481779 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 27 20:51:10.486135 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481783 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481785 2565 flags.go:64] FLAG: --experimental-mounter-path="" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481788 2565 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481791 2565 flags.go:64] FLAG: --fail-swap-on="true" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481794 2565 flags.go:64] FLAG: --feature-gates="" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481797 2565 flags.go:64] FLAG: --file-check-frequency="20s" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481800 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481803 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481806 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481809 2565 flags.go:64] FLAG: --healthz-port="10248" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481811 2565 flags.go:64] FLAG: --help="false" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481814 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-131-232.ec2.internal" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481817 2565 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481820 2565 flags.go:64] FLAG: --http-check-frequency="20s" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481822 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481826 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481829 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481831 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481834 2565 flags.go:64] FLAG: --image-service-endpoint="" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481837 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481840 2565 flags.go:64] FLAG: --kube-api-burst="100" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481842 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481845 2565 flags.go:64] FLAG: --kube-api-qps="50" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481848 2565 flags.go:64] FLAG: --kube-reserved="" Jul 27 20:51:10.486789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481850 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481853 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481856 2565 flags.go:64] FLAG: --kubelet-cgroups="" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481859 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481865 2565 flags.go:64] FLAG: --lock-file="" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481868 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481872 2565 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481875 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481880 2565 flags.go:64] FLAG: --log-json-split-stream="false" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481884 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481887 2565 flags.go:64] FLAG: --log-text-split-stream="false" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481890 2565 flags.go:64] FLAG: --logging-format="text" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481892 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481896 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481898 2565 flags.go:64] FLAG: --manifest-url="" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481901 2565 flags.go:64] FLAG: --manifest-url-header="" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481905 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481908 2565 flags.go:64] FLAG: --max-open-files="1000000" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481912 2565 flags.go:64] FLAG: --max-pods="110" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481915 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481918 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481921 2565 flags.go:64] FLAG: --memory-manager-policy="None" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481924 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481927 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481929 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 27 20:51:10.487382 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481932 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481939 2565 flags.go:64] FLAG: --node-status-max-images="50" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481942 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481945 2565 flags.go:64] FLAG: --oom-score-adj="-999" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481948 2565 flags.go:64] FLAG: --pod-cidr="" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481950 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481956 2565 flags.go:64] FLAG: --pod-manifest-path="" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481959 2565 flags.go:64] FLAG: --pod-max-pids="-1" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481962 2565 flags.go:64] FLAG: --pods-per-core="0" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481965 2565 flags.go:64] FLAG: --port="10250" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481967 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481970 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-070a57abcb6d40310" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481974 2565 flags.go:64] FLAG: --qos-reserved="" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481979 2565 flags.go:64] FLAG: --read-only-port="10255" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481982 2565 flags.go:64] FLAG: --register-node="true" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481985 2565 flags.go:64] FLAG: --register-with-taints="" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481988 2565 flags.go:64] FLAG: --registry-burst="10" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481992 2565 flags.go:64] FLAG: --registry-qps="5" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481994 2565 flags.go:64] FLAG: --reserved-cpus="" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.481997 2565 flags.go:64] FLAG: --reserved-memory="" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482001 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482004 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482007 2565 flags.go:64] FLAG: --rotate-certificates="false" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482010 2565 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482013 2565 flags.go:64] FLAG: --runonce="false" Jul 27 20:51:10.488027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482015 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482018 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482021 2565 flags.go:64] FLAG: --seccomp-default="false" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482024 2565 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482026 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482029 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482032 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482035 2565 flags.go:64] FLAG: --storage-driver-password="root" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482038 2565 flags.go:64] FLAG: --storage-driver-secure="false" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482040 2565 flags.go:64] FLAG: --storage-driver-table="stats" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482043 2565 flags.go:64] FLAG: --storage-driver-user="root" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482045 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482048 2565 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482051 2565 flags.go:64] FLAG: --system-cgroups="" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482054 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482058 2565 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482061 2565 flags.go:64] FLAG: --tls-cert-file="" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482064 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482068 2565 flags.go:64] FLAG: --tls-min-version="" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482072 2565 flags.go:64] FLAG: --tls-private-key-file="" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482076 2565 flags.go:64] FLAG: --topology-manager-policy="none" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482078 2565 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482081 2565 flags.go:64] FLAG: --topology-manager-scope="container" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482084 2565 flags.go:64] FLAG: --v="2" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482088 2565 flags.go:64] FLAG: --version="false" Jul 27 20:51:10.488651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482092 2565 flags.go:64] FLAG: --vmodule="" Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482096 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.482099 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482221 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482225 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482228 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482231 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482234 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482236 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482239 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482241 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482244 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482247 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482249 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482251 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482254 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482256 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482259 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482261 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482263 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 20:51:10.489285 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482266 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 20:51:10.489852 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482269 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 20:51:10.489852 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482271 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 20:51:10.489852 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482273 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 20:51:10.489852 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482276 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 20:51:10.489852 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482278 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 20:51:10.489852 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482282 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 20:51:10.489852 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482286 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 20:51:10.489852 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482288 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 20:51:10.489852 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482291 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 20:51:10.489852 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482293 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 20:51:10.489852 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482296 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 20:51:10.489852 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482298 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 20:51:10.489852 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482303 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 20:51:10.489852 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482305 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 20:51:10.489852 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482308 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 20:51:10.489852 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482311 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 20:51:10.489852 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482315 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 20:51:10.489852 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482318 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 20:51:10.489852 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482321 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482323 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482326 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482328 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482330 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482333 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482336 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482338 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482340 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482343 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482345 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482348 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482350 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482352 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482355 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482357 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482361 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482365 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482367 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482372 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 20:51:10.490310 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482376 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482379 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482382 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482385 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482388 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482390 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482393 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482396 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482398 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482401 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482403 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482406 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482409 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482411 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482413 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482416 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482418 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482421 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482424 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482440 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 20:51:10.490805 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482444 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 20:51:10.491290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482446 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 20:51:10.491290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482449 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 20:51:10.491290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482451 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 20:51:10.491290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482454 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 20:51:10.491290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482456 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 20:51:10.491290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482459 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 20:51:10.491290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482461 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 20:51:10.491290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482464 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 20:51:10.491290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482467 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 20:51:10.491290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482470 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 20:51:10.491290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482475 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 27 20:51:10.491290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482479 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 20:51:10.491290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482482 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 20:51:10.491290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482485 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 20:51:10.491290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482487 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 20:51:10.491290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482490 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 20:51:10.491290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482492 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 20:51:10.491290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482495 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 20:51:10.491290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482497 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 20:51:10.491769 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482500 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 20:51:10.491769 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482503 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 20:51:10.491769 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.482505 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 20:51:10.491769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.483153 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 27 20:51:10.491769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.490277 2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 27 20:51:10.491769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.490291 2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 27 20:51:10.491769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.490310 2565 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 27 20:51:10.491769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.490316 2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 27 20:51:10.491769 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490375 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 20:51:10.491769 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490380 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 27 20:51:10.491769 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490383 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 20:51:10.491769 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490386 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 20:51:10.491769 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490389 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 20:51:10.491769 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490393 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490396 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490398 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490401 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490403 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490406 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490408 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490411 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490414 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490416 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490419 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490422 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490425 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490441 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490443 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490446 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490448 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490451 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490453 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490457 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 20:51:10.492109 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490459 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 20:51:10.492608 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490462 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 20:51:10.492608 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490464 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 20:51:10.492608 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490466 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 20:51:10.492608 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490469 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 20:51:10.492608 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490471 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 20:51:10.492608 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490473 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 20:51:10.492608 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490476 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 20:51:10.492608 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490479 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 20:51:10.492608 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490483 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 20:51:10.492608 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490486 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 20:51:10.492608 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490488 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 20:51:10.492608 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490491 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 20:51:10.492608 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490494 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 20:51:10.492608 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490496 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 20:51:10.492608 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490499 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 20:51:10.492608 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490501 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 20:51:10.492608 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490503 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 20:51:10.492608 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490507 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 20:51:10.492608 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490511 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 20:51:10.493060 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490514 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 20:51:10.493060 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490516 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 20:51:10.493060 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490518 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 20:51:10.493060 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490521 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 20:51:10.493060 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490524 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 20:51:10.493060 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490526 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 20:51:10.493060 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490529 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 20:51:10.493060 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490531 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 20:51:10.493060 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490534 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 20:51:10.493060 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490536 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 20:51:10.493060 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490540 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 20:51:10.493060 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490544 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 20:51:10.493060 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490547 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 20:51:10.493060 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490550 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 20:51:10.493060 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490552 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 20:51:10.493060 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490555 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 20:51:10.493060 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490557 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 20:51:10.493060 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490560 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 20:51:10.493060 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490563 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490565 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490568 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490570 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490573 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490575 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490578 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490580 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490583 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490585 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490588 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490590 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490593 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490595 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490598 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490600 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490603 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490605 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490608 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490611 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 20:51:10.493677 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490613 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 20:51:10.494156 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490616 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 20:51:10.494156 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490618 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 20:51:10.494156 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490620 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 20:51:10.494156 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490622 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 20:51:10.494156 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490625 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 20:51:10.494156 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490627 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 20:51:10.494156 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490630 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 20:51:10.494156 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490632 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 20:51:10.494156 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490635 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 20:51:10.494156 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490637 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 20:51:10.494156 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490640 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 20:51:10.494156 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490642 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 20:51:10.494156 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490644 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 20:51:10.494156 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490647 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 20:51:10.494156 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490649 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 20:51:10.494532 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.490654 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 27 20:51:10.494532 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490767 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 20:51:10.494532 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490771 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 20:51:10.494532 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490774 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 20:51:10.494532 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490778 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 20:51:10.494532 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490782 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 20:51:10.494532 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490786 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 20:51:10.494532 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490789 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 20:51:10.494532 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490791 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 20:51:10.494532 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490794 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 20:51:10.494532 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490796 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 20:51:10.494532 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490799 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 20:51:10.494532 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490801 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 20:51:10.494532 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490804 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 20:51:10.494532 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490806 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490809 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490811 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490814 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490816 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490819 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490821 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490824 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490827 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490829 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490832 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490835 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490837 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490840 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490842 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490845 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490848 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490852 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490854 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490857 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 27 20:51:10.494902 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490859 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490862 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490864 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490867 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490870 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490872 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490875 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490877 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490880 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490882 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490885 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490887 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490890 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490892 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490895 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490897 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490900 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490902 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490904 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490907 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 20:51:10.495381 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490909 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 20:51:10.495862 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490911 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 20:51:10.495862 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490914 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 20:51:10.495862 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490917 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 20:51:10.495862 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490919 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 20:51:10.495862 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490921 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 20:51:10.495862 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490924 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 20:51:10.495862 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490926 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 20:51:10.495862 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490929 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 20:51:10.495862 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490931 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 20:51:10.495862 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490934 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 20:51:10.495862 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490936 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 20:51:10.495862 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490939 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 20:51:10.495862 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490941 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 20:51:10.495862 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490943 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 20:51:10.495862 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490946 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 20:51:10.495862 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490949 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 20:51:10.495862 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490951 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 20:51:10.495862 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490955 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 20:51:10.495862 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490958 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490961 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490964 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490967 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490969 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490972 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490974 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490977 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490979 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490981 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490984 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490986 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490989 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490991 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490994 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490996 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.490999 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.491002 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.491004 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.491006 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 20:51:10.496326 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.491009 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 20:51:10.496815 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.491011 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 20:51:10.496815 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.491014 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 20:51:10.496815 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.491016 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 20:51:10.496815 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.491019 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 20:51:10.496815 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.491021 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 20:51:10.496815 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:10.491023 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 20:51:10.496815 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.491030 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 27 20:51:10.496815 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.491129 2565 server.go:962] "Client rotation is on, will bootstrap in background" Jul 27 20:51:10.496815 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.493304 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 27 20:51:10.496815 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.494822 2565 server.go:1019] "Starting client certificate rotation" Jul 27 20:51:10.496815 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.494917 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 27 20:51:10.496815 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.494953 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 27 20:51:10.497959 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.497945 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 27 20:51:10.499113 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.499098 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 27 20:51:10.511244 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.511218 2565 log.go:25] "Validated CRI v1 runtime API" Jul 27 20:51:10.513518 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.513499 2565 log.go:25] "Validated CRI v1 image API" Jul 27 20:51:10.514328 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.514307 2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 27 20:51:10.517653 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.517634 2565 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8b30903c-8110-4047-ab4a-a854a35ae9eb:/dev/nvme0n1p3 a6471758-62be-4c3f-bba5-c9db0d0573ff:/dev/nvme0n1p4] Jul 27 20:51:10.517715 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.517653 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 27 20:51:10.521868 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.521731 2565 manager.go:216] Machine: {Timestamp:2026-07-27 20:51:10.521218531 +0000 UTC m=+0.380141465 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3112035 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26f059b63f570faf2434e768ddc541 SystemUUID:ec26f059-b63f-570f-af24-34e768ddc541 BootID:9be49b2c-12cc-4e70-a971-0aea502a9c2f Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1f:7f:45:64:4b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1f:7f:45:64:4b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:9a:a7:cc:9a:8a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 27 20:51:10.521948 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.521866 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 27 20:51:10.521981 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.521952 2565 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 27 20:51:10.523711 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.523688 2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 27 20:51:10.523847 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.523715 2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-232.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 27 20:51:10.524467 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.524457 2565 topology_manager.go:138] "Creating topology manager with none policy" Jul 27 20:51:10.524500 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.524470 2565 container_manager_linux.go:309] "Creating device plugin manager" Jul 27 20:51:10.524500 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.524479 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 27 20:51:10.525308 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.525297 2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 27 20:51:10.526251 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.526235 2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 27 20:51:10.528573 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.528550 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 27 20:51:10.528800 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.528790 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 27 20:51:10.528941 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.528931 2565 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 27 20:51:10.528970 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.528963 2565 kubelet.go:486] "Attempting to sync node with API server" Jul 27 20:51:10.528996 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.528972 2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 27 20:51:10.528996 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.528987 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 27 20:51:10.528996 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.528995 2565 kubelet.go:398] "Adding apiserver pod source" Jul 27 20:51:10.529077 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.529004 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 27 20:51:10.529372 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.529361 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 27 20:51:10.529417 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.529385 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 27 20:51:10.532030 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.532011 2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 27 20:51:10.534154 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.534137 2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 27 20:51:10.534244 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.534176 2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 27 20:51:10.536667 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.536652 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 27 20:51:10.536737 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.536676 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 27 20:51:10.536737 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.536685 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 27 20:51:10.536737 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.536693 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 27 20:51:10.536737 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.536705 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 27 20:51:10.536737 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.536714 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 27 20:51:10.536737 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.536727 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 27 20:51:10.536737 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.536736 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 27 20:51:10.536975 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.536747 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 27 20:51:10.536975 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.536757 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 27 20:51:10.536975 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.536771 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 27 20:51:10.536975 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.536779 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 27 20:51:10.537883 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.537871 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 27 20:51:10.537946 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.537887 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 27 20:51:10.540249 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.540230 2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-232.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 27 20:51:10.540328 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.540257 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-232.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 27 20:51:10.540328 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.540256 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 27 20:51:10.541074 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.541060 2565 server.go:1268] "Started kubelet" Jul 27 20:51:10.541148 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.541109 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 27 20:51:10.541198 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.541129 2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 27 20:51:10.541239 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.541202 2565 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 27 20:51:10.542065 ip-10-0-131-232 systemd[1]: Started Kubernetes Kubelet. Jul 27 20:51:10.542319 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.542305 2565 server.go:310] "Adding debug handlers to kubelet server" Jul 27 20:51:10.543524 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.543412 2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 27 20:51:10.547654 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.547633 2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 27 20:51:10.552395 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.552380 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 27 20:51:10.552462 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.552447 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 27 20:51:10.552531 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.552513 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-232.ec2.internal\" not found" Jul 27 20:51:10.553141 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.553114 2565 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 27 20:51:10.553141 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.553133 2565 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 27 20:51:10.553280 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.553113 2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 27 20:51:10.553280 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.553218 2565 factory.go:55] Registering systemd factory Jul 27 20:51:10.553280 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.553227 2565 reconstruct.go:97] "Volume reconstruction finished" Jul 27 20:51:10.553280 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.553238 2565 reconciler.go:29] "Reconciler: start to sync state" Jul 27 20:51:10.553280 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.553232 2565 factory.go:223] Registration of the systemd container factory successfully Jul 27 20:51:10.553567 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.553549 2565 factory.go:153] Registering CRI-O factory Jul 27 20:51:10.553567 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.553566 2565 factory.go:223] Registration of the crio container factory successfully Jul 27 20:51:10.553670 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.553614 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 27 20:51:10.553670 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.553638 2565 factory.go:103] Registering Raw factory Jul 27 20:51:10.553670 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.553652 2565 manager.go:1196] Started watching for new ooms in manager Jul 27 20:51:10.554077 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.554063 2565 manager.go:349] Starting recovery of all containers Jul 27 20:51:10.556523 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.556492 2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-232.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 27 20:51:10.557487 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.557464 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 27 20:51:10.557616 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.556494 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-232.ec2.internal.18c640ba25c96e79 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-232.ec2.internal,UID:ip-10-0-131-232.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-232.ec2.internal,},FirstTimestamp:2026-07-27 20:51:10.541033081 +0000 UTC m=+0.399956015,LastTimestamp:2026-07-27 20:51:10.541033081 +0000 UTC m=+0.399956015,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-232.ec2.internal,}" Jul 27 20:51:10.562508 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.562493 2565 manager.go:354] Recovery completed Jul 27 20:51:10.566461 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.566447 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 20:51:10.568056 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.568041 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasSufficientMemory" Jul 27 20:51:10.568113 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.568067 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 20:51:10.568113 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.568078 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasSufficientPID" Jul 27 20:51:10.568569 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.568556 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 27 20:51:10.568569 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.568568 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 27 20:51:10.568668 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.568582 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 27 20:51:10.568668 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.568621 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 20:51:10.569194 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.569179 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasSufficientMemory" Jul 27 20:51:10.569252 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.569205 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 20:51:10.569252 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.569217 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasSufficientPID" Jul 27 20:51:10.570623 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.570532 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-232.ec2.internal.18c640ba2765c414 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-232.ec2.internal,UID:ip-10-0-131-232.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-232.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-232.ec2.internal,},FirstTimestamp:2026-07-27 20:51:10.568055828 +0000 UTC m=+0.426978761,LastTimestamp:2026-07-27 20:51:10.568055828 +0000 UTC m=+0.426978761,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-232.ec2.internal,}" Jul 27 20:51:10.570692 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.570618 2565 policy_none.go:49] "None policy: Start" Jul 27 20:51:10.570726 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.570697 2565 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 27 20:51:10.570726 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.570707 2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 27 20:51:10.571709 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.571699 2565 policy_none.go:47] "Start" Jul 27 20:51:10.580061 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.580000 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-232.ec2.internal.18c640ba276603f0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-232.ec2.internal,UID:ip-10-0-131-232.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-232.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-232.ec2.internal,},FirstTimestamp:2026-07-27 20:51:10.568072176 +0000 UTC m=+0.426995110,LastTimestamp:2026-07-27 20:51:10.568072176 +0000 UTC m=+0.426995110,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-232.ec2.internal,}" Jul 27 20:51:10.590140 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.589968 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dtfxr" Jul 27 20:51:10.590994 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.589977 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-232.ec2.internal.18c640ba276629b0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-232.ec2.internal,UID:ip-10-0-131-232.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-232.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-232.ec2.internal,},FirstTimestamp:2026-07-27 20:51:10.56808184 +0000 UTC m=+0.427004773,LastTimestamp:2026-07-27 20:51:10.56808184 +0000 UTC m=+0.427004773,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-232.ec2.internal,}" Jul 27 20:51:10.598151 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.598125 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dtfxr" Jul 27 20:51:10.604334 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.604268 2565 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-232.ec2.internal.18c640ba2765c414\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-232.ec2.internal.18c640ba2765c414 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-232.ec2.internal,UID:ip-10-0-131-232.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-232.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-232.ec2.internal,},FirstTimestamp:2026-07-27 20:51:10.568055828 +0000 UTC m=+0.426978761,LastTimestamp:2026-07-27 20:51:10.569194348 +0000 UTC m=+0.428117286,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-232.ec2.internal,}" Jul 27 20:51:10.614110 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.614091 2565 manager.go:337] "Starting Device Plugin manager" Jul 27 20:51:10.614194 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.614141 2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 27 20:51:10.614194 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.614154 2565 server.go:87] "Starting device plugin registration server" Jul 27 20:51:10.614823 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.614389 2565 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 27 20:51:10.614823 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.614402 2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 27 20:51:10.614823 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.614711 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 27 20:51:10.614823 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.614772 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 27 20:51:10.614823 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.614778 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 27 20:51:10.615749 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.615707 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 27 20:51:10.615838 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.615764 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-232.ec2.internal\" not found" Jul 27 20:51:10.708801 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.708766 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 27 20:51:10.710120 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.710104 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 27 20:51:10.710179 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.710144 2565 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 27 20:51:10.710179 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.710168 2565 kubelet.go:2448] "Starting kubelet main sync loop" Jul 27 20:51:10.710259 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.710201 2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 27 20:51:10.712525 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.712500 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 20:51:10.715233 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.715216 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 20:51:10.716164 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.716145 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasSufficientMemory" Jul 27 20:51:10.716257 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.716179 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 20:51:10.716257 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.716193 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasSufficientPID" Jul 27 20:51:10.716257 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.716218 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-232.ec2.internal" Jul 27 20:51:10.724790 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.724747 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-232.ec2.internal" Jul 27 20:51:10.724790 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.724769 2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-232.ec2.internal\": node \"ip-10-0-131-232.ec2.internal\" not found" Jul 27 20:51:10.761397 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.761370 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-232.ec2.internal\" not found" Jul 27 20:51:10.810734 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.810688 2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-232.ec2.internal"] Jul 27 20:51:10.810810 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.810778 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 20:51:10.811779 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.811761 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasSufficientMemory" Jul 27 20:51:10.811853 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.811796 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 20:51:10.811853 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.811808 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasSufficientPID" Jul 27 20:51:10.813203 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.813191 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 20:51:10.813312 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.813299 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal" Jul 27 20:51:10.813357 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.813327 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 20:51:10.813991 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.813973 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasSufficientMemory" Jul 27 20:51:10.814089 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.814002 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 20:51:10.814089 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.814012 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasSufficientPID" Jul 27 20:51:10.814916 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.814895 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasSufficientMemory" Jul 27 20:51:10.815004 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.814925 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 20:51:10.815004 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.814935 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasSufficientPID" Jul 27 20:51:10.816252 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.816239 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-232.ec2.internal" Jul 27 20:51:10.816296 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.816265 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 20:51:10.816911 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.816895 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasSufficientMemory" Jul 27 20:51:10.816982 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.816924 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 20:51:10.816982 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.816935 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasSufficientPID" Jul 27 20:51:10.831009 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.830988 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-232.ec2.internal\" not found" node="ip-10-0-131-232.ec2.internal" Jul 27 20:51:10.834919 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.834902 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-232.ec2.internal\" not found" node="ip-10-0-131-232.ec2.internal" Jul 27 20:51:10.854860 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.854831 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8c2f8ede239810f1da80ef8aeebedd8a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal\" (UID: \"8c2f8ede239810f1da80ef8aeebedd8a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal" Jul 27 20:51:10.854860 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.854858 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c2f8ede239810f1da80ef8aeebedd8a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal\" (UID: \"8c2f8ede239810f1da80ef8aeebedd8a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal" Jul 27 20:51:10.855034 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.854876 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9233b835deeeffbb2950b8b4fe9f0d75-config\") pod \"kube-apiserver-proxy-ip-10-0-131-232.ec2.internal\" (UID: \"9233b835deeeffbb2950b8b4fe9f0d75\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-232.ec2.internal" Jul 27 20:51:10.861939 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.861921 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-232.ec2.internal\" not found" Jul 27 20:51:10.955297 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.955258 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8c2f8ede239810f1da80ef8aeebedd8a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal\" (UID: \"8c2f8ede239810f1da80ef8aeebedd8a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal" Jul 27 20:51:10.955297 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.955296 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c2f8ede239810f1da80ef8aeebedd8a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal\" (UID: \"8c2f8ede239810f1da80ef8aeebedd8a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal" Jul 27 20:51:10.955535 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.955313 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9233b835deeeffbb2950b8b4fe9f0d75-config\") pod \"kube-apiserver-proxy-ip-10-0-131-232.ec2.internal\" (UID: \"9233b835deeeffbb2950b8b4fe9f0d75\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-232.ec2.internal" Jul 27 20:51:10.955535 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.955364 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9233b835deeeffbb2950b8b4fe9f0d75-config\") pod \"kube-apiserver-proxy-ip-10-0-131-232.ec2.internal\" (UID: \"9233b835deeeffbb2950b8b4fe9f0d75\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-232.ec2.internal" Jul 27 20:51:10.955535 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.955368 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c2f8ede239810f1da80ef8aeebedd8a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal\" (UID: \"8c2f8ede239810f1da80ef8aeebedd8a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal" Jul 27 20:51:10.955535 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:10.955369 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8c2f8ede239810f1da80ef8aeebedd8a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal\" (UID: \"8c2f8ede239810f1da80ef8aeebedd8a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal" Jul 27 20:51:10.962339 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:10.962319 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-232.ec2.internal\" not found" Jul 27 20:51:11.063275 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:11.063208 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-232.ec2.internal\" not found" Jul 27 20:51:11.134437 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:11.134407 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal" Jul 27 20:51:11.138676 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:11.138486 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-232.ec2.internal" Jul 27 20:51:11.163561 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:11.163538 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-232.ec2.internal\" not found" Jul 27 20:51:11.264107 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:11.264065 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-232.ec2.internal\" not found" Jul 27 20:51:11.364555 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:11.364528 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-232.ec2.internal\" not found" Jul 27 20:51:11.465256 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:11.465212 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-232.ec2.internal\" not found" Jul 27 20:51:11.494384 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:11.494359 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 27 20:51:11.494790 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:11.494510 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 27 20:51:11.514067 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:11.514045 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 20:51:11.565861 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:11.565839 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-232.ec2.internal\" not found" Jul 27 20:51:11.603017 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:11.602982 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-26 20:46:11 +0000 UTC" deadline="2028-04-27 05:01:44.8205582 +0000 UTC" Jul 27 20:51:11.603017 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:11.603011 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15344h10m33.217550483s" Jul 27 20:51:11.619314 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:11.619269 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 20:51:11.620132 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:11.620110 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasSufficientMemory" Jul 27 20:51:11.620232 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:11.620138 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 20:51:11.620232 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:11.620149 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeHasSufficientPID" Jul 27 20:51:11.666483 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:11.666419 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-232.ec2.internal\" not found" Jul 27 20:51:11.749521 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:11.749493 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9233b835deeeffbb2950b8b4fe9f0d75.slice/crio-a9df9501ae70673fad04b812aef08ae35a041a892ca7617698290757e617803f WatchSource:0}: Error finding container a9df9501ae70673fad04b812aef08ae35a041a892ca7617698290757e617803f: Status 404 returned error can't find the container with id a9df9501ae70673fad04b812aef08ae35a041a892ca7617698290757e617803f Jul 27 20:51:11.750058 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:11.750027 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c2f8ede239810f1da80ef8aeebedd8a.slice/crio-a033f4fb947d57fd8da7678b28c643e4c9df021828bf2a77152013da24b8fd56 WatchSource:0}: Error finding container a033f4fb947d57fd8da7678b28c643e4c9df021828bf2a77152013da24b8fd56: Status 404 returned error can't find the container with id a033f4fb947d57fd8da7678b28c643e4c9df021828bf2a77152013da24b8fd56 Jul 27 20:51:11.755329 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:11.755311 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 20:51:11.767020 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:11.767003 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-232.ec2.internal\" not found" Jul 27 20:51:11.852578 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:11.852551 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 20:51:11.852718 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:11.852650 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal" Jul 27 20:51:11.873170 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:11.873119 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 27 20:51:11.875984 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:11.875969 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-232.ec2.internal" Jul 27 20:51:11.887017 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:11.887001 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 27 20:51:13.367817 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.367792 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 20:51:13.675396 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.675362 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 20:51:13.865404 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.865369 2565 apiserver.go:52] "Watching apiserver" Jul 27 20:51:13.873284 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.873256 2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 27 20:51:13.873621 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.873595 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh","openshift-dns/node-resolver-lbt9p","openshift-image-registry/node-ca-74fh5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal","openshift-multus/multus-additional-cni-plugins-h9lsk","openshift-multus/network-metrics-daemon-wk44n","openshift-network-operator/iptables-alerter-nxnxn","kube-system/global-pull-secret-syncer-qzsr7","kube-system/konnectivity-agent-td7kp","kube-system/kube-apiserver-proxy-ip-10-0-131-232.ec2.internal","openshift-cluster-node-tuning-operator/tuned-njk5z","openshift-multus/multus-6vdnr","openshift-network-diagnostics/network-check-target-zt78l","openshift-ovn-kubernetes/ovnkube-node-9gbjx"] Jul 27 20:51:13.875600 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.875288 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:13.875600 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:13.875399 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zt78l" podUID="8add67af-ae5d-4d84-b040-7d8cc145e665" Jul 27 20:51:13.876851 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.876832 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lbt9p" Jul 27 20:51:13.878514 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.878122 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-74fh5" Jul 27 20:51:13.879419 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.879399 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 27 20:51:13.879511 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.879424 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 27 20:51:13.879570 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.879404 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xrs2s\"" Jul 27 20:51:13.879744 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.879726 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:13.880687 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.880670 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 27 20:51:13.880778 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.880698 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 27 20:51:13.880991 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.880973 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 27 20:51:13.880991 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.880987 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tr2bm\"" Jul 27 20:51:13.881218 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.881196 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:13.882349 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.882320 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:13.882429 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:13.882397 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wk44n" podUID="668f5b4e-867c-47c2-b868-cb3f79c14f2a" Jul 27 20:51:13.883098 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.883077 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 27 20:51:13.883685 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.883665 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 27 20:51:13.883867 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.883848 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 27 20:51:13.883958 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.883934 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nxnxn" Jul 27 20:51:13.884164 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.884146 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gtnqd\"" Jul 27 20:51:13.884667 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.884422 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vjpkp\"" Jul 27 20:51:13.884667 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.884579 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 27 20:51:13.884939 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.884916 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 27 20:51:13.885027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.884948 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 27 20:51:13.885027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.884965 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 27 20:51:13.885027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.884996 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 27 20:51:13.885317 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.885286 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:13.885395 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:13.885366 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qzsr7" podUID="86fb8fed-0e82-44c5-9c08-08a9b4d8eea3" Jul 27 20:51:13.887331 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.886615 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 27 20:51:13.887331 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.886783 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-td7kp" Jul 27 20:51:13.887331 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.886832 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vkptk\"" Jul 27 20:51:13.887331 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.886981 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 27 20:51:13.887626 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.887609 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 27 20:51:13.889226 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.888621 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.890211 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.890192 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:13.890472 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.890452 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 27 20:51:13.890910 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.890894 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 27 20:51:13.890983 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.890970 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 27 20:51:13.891128 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.891116 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-z7xs7\"" Jul 27 20:51:13.891173 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.891127 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 27 20:51:13.891658 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.891641 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 27 20:51:13.891744 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.891675 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nwqx2\"" Jul 27 20:51:13.891866 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.891851 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 27 20:51:13.893220 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.893190 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6vdnr" Jul 27 20:51:13.894943 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.894917 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 27 20:51:13.895035 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.894984 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4zr2m\"" Jul 27 20:51:13.895099 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.895040 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 27 20:51:13.895303 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.895285 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 27 20:51:13.895361 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.895311 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 27 20:51:13.895657 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.895640 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 27 20:51:13.895965 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.895947 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 27 20:51:13.896043 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.895991 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-c7lql\"" Jul 27 20:51:13.899669 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.899647 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9ws7\" (UniqueName: \"kubernetes.io/projected/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-kube-api-access-r9ws7\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:13.899753 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.899682 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-etc-openvswitch\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.899753 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.899707 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-log-socket\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.899866 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.899759 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-cni-netd\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.899866 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.899795 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78a12024-7c5c-4f1c-a15d-58876f88ee45-hosts-file\") pod \"node-resolver-lbt9p\" (UID: \"78a12024-7c5c-4f1c-a15d-58876f88ee45\") " pod="openshift-dns/node-resolver-lbt9p" Jul 27 20:51:13.899866 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.899817 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78a12024-7c5c-4f1c-a15d-58876f88ee45-tmp-dir\") pod \"node-resolver-lbt9p\" (UID: \"78a12024-7c5c-4f1c-a15d-58876f88ee45\") " pod="openshift-dns/node-resolver-lbt9p" Jul 27 20:51:13.899866 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.899840 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-lib-modules\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:13.899866 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.899862 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-var-lib-kubelet\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:13.900041 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.899886 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-var-lib-openvswitch\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.900041 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.899908 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4d712e4f-4317-4915-a4d0-2e02a6154d8c-env-overrides\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.900041 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.899930 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-etc-sysctl-conf\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:13.900041 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.899952 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-run-systemd\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.900041 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.899973 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed75d226-bf8c-4148-8554-135bc455a794-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:13.900041 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.899996 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed75d226-bf8c-4148-8554-135bc455a794-etc-selinux\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:13.900041 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900019 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4d712e4f-4317-4915-a4d0-2e02a6154d8c-ovnkube-config\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.900385 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900052 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/727b584c-bed9-4780-a327-b84b809591a9-host-slash\") pod \"iptables-alerter-nxnxn\" (UID: \"727b584c-bed9-4780-a327-b84b809591a9\") " pod="openshift-network-operator/iptables-alerter-nxnxn" Jul 27 20:51:13.900385 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900076 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-host\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:13.900385 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900100 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3705087b-83f8-4e75-84e2-ce01936e4657-host\") pod \"node-ca-74fh5\" (UID: \"3705087b-83f8-4e75-84e2-ce01936e4657\") " pod="openshift-image-registry/node-ca-74fh5" Jul 27 20:51:13.900385 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900126 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8a1339c-f7dc-4b4e-9a09-c82402afe736-konnectivity-ca\") pod \"konnectivity-agent-td7kp\" (UID: \"e8a1339c-f7dc-4b4e-9a09-c82402afe736\") " pod="kube-system/konnectivity-agent-td7kp" Jul 27 20:51:13.900385 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900152 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed75d226-bf8c-4148-8554-135bc455a794-socket-dir\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:13.900385 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900178 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:13.900385 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900200 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-etc-sysconfig\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:13.900385 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900277 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-etc-kubernetes\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:13.900385 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900312 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-run\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:13.900385 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900336 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3705087b-83f8-4e75-84e2-ce01936e4657-serviceca\") pod \"node-ca-74fh5\" (UID: \"3705087b-83f8-4e75-84e2-ce01936e4657\") " pod="openshift-image-registry/node-ca-74fh5" Jul 27 20:51:13.900385 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900369 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed75d226-bf8c-4148-8554-135bc455a794-device-dir\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:13.900385 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900391 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8a1339c-f7dc-4b4e-9a09-c82402afe736-agent-certs\") pod \"konnectivity-agent-td7kp\" (UID: \"e8a1339c-f7dc-4b4e-9a09-c82402afe736\") " pod="kube-system/konnectivity-agent-td7kp" Jul 27 20:51:13.900905 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900411 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk2g6\" (UniqueName: \"kubernetes.io/projected/3705087b-83f8-4e75-84e2-ce01936e4657-kube-api-access-qk2g6\") pod \"node-ca-74fh5\" (UID: \"3705087b-83f8-4e75-84e2-ce01936e4657\") " pod="openshift-image-registry/node-ca-74fh5" Jul 27 20:51:13.900905 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900436 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-run-ovn-kubernetes\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.900905 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900460 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znkxb\" (UniqueName: \"kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb\") pod \"network-check-target-zt78l\" (UID: \"8add67af-ae5d-4d84-b040-7d8cc145e665\") " pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:13.900905 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900483 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5srj\" (UniqueName: \"kubernetes.io/projected/727b584c-bed9-4780-a327-b84b809591a9-kube-api-access-c5srj\") pod \"iptables-alerter-nxnxn\" (UID: \"727b584c-bed9-4780-a327-b84b809591a9\") " pod="openshift-network-operator/iptables-alerter-nxnxn" Jul 27 20:51:13.900905 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900505 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-etc-sysctl-d\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:13.900905 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900528 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-systemd-units\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.900905 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900550 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-cni-binary-copy\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:13.900905 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900575 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-os-release\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:13.900905 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900602 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-dbus\") pod \"global-pull-secret-syncer-qzsr7\" (UID: \"86fb8fed-0e82-44c5-9c08-08a9b4d8eea3\") " pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:13.900905 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900626 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4d712e4f-4317-4915-a4d0-2e02a6154d8c-ovn-node-metrics-cert\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.900905 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900708 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw5xb\" (UniqueName: \"kubernetes.io/projected/ed75d226-bf8c-4148-8554-135bc455a794-kube-api-access-kw5xb\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:13.900905 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900734 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-kubelet-config\") pod \"global-pull-secret-syncer-qzsr7\" (UID: \"86fb8fed-0e82-44c5-9c08-08a9b4d8eea3\") " pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:13.900905 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900757 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-run-ovn\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.900905 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900816 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpkwr\" (UniqueName: \"kubernetes.io/projected/464f92c8-5626-4fef-9eff-137f0c713279-kube-api-access-jpkwr\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:13.900905 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900838 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-run-netns\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.900905 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900878 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4d712e4f-4317-4915-a4d0-2e02a6154d8c-ovnkube-script-lib\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.901443 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900897 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbdpl\" (UniqueName: \"kubernetes.io/projected/668f5b4e-867c-47c2-b868-cb3f79c14f2a-kube-api-access-qbdpl\") pod \"network-metrics-daemon-wk44n\" (UID: \"668f5b4e-867c-47c2-b868-cb3f79c14f2a\") " pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:13.901443 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900929 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-cni-bin\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.901443 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900948 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs\") pod \"network-metrics-daemon-wk44n\" (UID: \"668f5b4e-867c-47c2-b868-cb3f79c14f2a\") " pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:13.901443 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.900973 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/464f92c8-5626-4fef-9eff-137f0c713279-etc-tuned\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:13.901443 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.901003 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-node-log\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.901443 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.901054 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/464f92c8-5626-4fef-9eff-137f0c713279-tmp\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:13.901443 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.901085 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-system-cni-dir\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:13.901443 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.901108 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-etc-modprobe-d\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:13.901443 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.901130 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-etc-systemd\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:13.901443 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.901159 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed75d226-bf8c-4148-8554-135bc455a794-registration-dir\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:13.901443 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.901180 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-cnibin\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:13.901443 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.901204 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.901443 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.901225 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret\") pod \"global-pull-secret-syncer-qzsr7\" (UID: \"86fb8fed-0e82-44c5-9c08-08a9b4d8eea3\") " pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:13.901443 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.901262 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-kubelet\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.901443 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.901286 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-run-openvswitch\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.901443 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.901309 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5gzj\" (UniqueName: \"kubernetes.io/projected/4d712e4f-4317-4915-a4d0-2e02a6154d8c-kube-api-access-m5gzj\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.901944 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.901338 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/727b584c-bed9-4780-a327-b84b809591a9-iptables-alerter-script\") pod \"iptables-alerter-nxnxn\" (UID: \"727b584c-bed9-4780-a327-b84b809591a9\") " pod="openshift-network-operator/iptables-alerter-nxnxn" Jul 27 20:51:13.901944 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.901362 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-sys\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:13.901944 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.901394 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:13.901944 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.901415 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed75d226-bf8c-4148-8554-135bc455a794-sys-fs\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:13.901944 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.901429 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:13.901944 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.901525 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-slash\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:13.901944 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.901564 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sflw2\" (UniqueName: \"kubernetes.io/projected/78a12024-7c5c-4f1c-a15d-58876f88ee45-kube-api-access-sflw2\") pod \"node-resolver-lbt9p\" (UID: \"78a12024-7c5c-4f1c-a15d-58876f88ee45\") " pod="openshift-dns/node-resolver-lbt9p" Jul 27 20:51:13.906062 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.905919 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 27 20:51:13.927927 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.927027 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9s9xt" Jul 27 20:51:13.936140 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.935838 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9s9xt" Jul 27 20:51:13.990078 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:13.990055 2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 27 20:51:14.002148 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002125 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed75d226-bf8c-4148-8554-135bc455a794-registration-dir\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:14.002284 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002154 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-cnibin\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:14.002284 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002171 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.002284 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002188 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret\") pod \"global-pull-secret-syncer-qzsr7\" (UID: \"86fb8fed-0e82-44c5-9c08-08a9b4d8eea3\") " pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:14.002284 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002211 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-kubelet\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.002284 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002249 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-run-openvswitch\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.002284 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002263 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-cnibin\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:14.002284 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002267 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed75d226-bf8c-4148-8554-135bc455a794-registration-dir\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:14.002284 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002273 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.002769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002274 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5gzj\" (UniqueName: \"kubernetes.io/projected/4d712e4f-4317-4915-a4d0-2e02a6154d8c-kube-api-access-m5gzj\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.002769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002314 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-kubelet\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.002769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002323 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/727b584c-bed9-4780-a327-b84b809591a9-iptables-alerter-script\") pod \"iptables-alerter-nxnxn\" (UID: \"727b584c-bed9-4780-a327-b84b809591a9\") " pod="openshift-network-operator/iptables-alerter-nxnxn" Jul 27 20:51:14.002769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002328 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-run-openvswitch\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.002769 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:14.002335 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 20:51:14.002769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002351 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-sys\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.002769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002380 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:14.002769 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:14.002436 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret podName:86fb8fed-0e82-44c5-9c08-08a9b4d8eea3 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:14.502384249 +0000 UTC m=+3.025501956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret") pod "global-pull-secret-syncer-qzsr7" (UID: "86fb8fed-0e82-44c5-9c08-08a9b4d8eea3") : object "kube-system"/"original-pull-secret" not registered Jul 27 20:51:14.002769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002453 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-sys\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.002769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002467 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-hostroot\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.002769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002494 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed75d226-bf8c-4148-8554-135bc455a794-sys-fs\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:14.002769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002520 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:14.002769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002542 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-slash\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.002769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002570 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sflw2\" (UniqueName: \"kubernetes.io/projected/78a12024-7c5c-4f1c-a15d-58876f88ee45-kube-api-access-sflw2\") pod \"node-resolver-lbt9p\" (UID: \"78a12024-7c5c-4f1c-a15d-58876f88ee45\") " pod="openshift-dns/node-resolver-lbt9p" Jul 27 20:51:14.002769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002596 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9ws7\" (UniqueName: \"kubernetes.io/projected/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-kube-api-access-r9ws7\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:14.002769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002620 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-etc-openvswitch\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.002769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002626 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-slash\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.003583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002650 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-log-socket\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.003583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002666 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-cni-netd\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.003583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002668 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-etc-openvswitch\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.003583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002594 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed75d226-bf8c-4148-8554-135bc455a794-sys-fs\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:14.003583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002681 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78a12024-7c5c-4f1c-a15d-58876f88ee45-hosts-file\") pod \"node-resolver-lbt9p\" (UID: \"78a12024-7c5c-4f1c-a15d-58876f88ee45\") " pod="openshift-dns/node-resolver-lbt9p" Jul 27 20:51:14.003583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002707 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/78a12024-7c5c-4f1c-a15d-58876f88ee45-hosts-file\") pod \"node-resolver-lbt9p\" (UID: \"78a12024-7c5c-4f1c-a15d-58876f88ee45\") " pod="openshift-dns/node-resolver-lbt9p" Jul 27 20:51:14.003583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002727 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78a12024-7c5c-4f1c-a15d-58876f88ee45-tmp-dir\") pod \"node-resolver-lbt9p\" (UID: \"78a12024-7c5c-4f1c-a15d-58876f88ee45\") " pod="openshift-dns/node-resolver-lbt9p" Jul 27 20:51:14.003583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002752 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-lib-modules\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.003583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002776 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-var-lib-kubelet\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.003583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002797 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-var-lib-openvswitch\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.003583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002811 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:14.003583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002824 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4d712e4f-4317-4915-a4d0-2e02a6154d8c-env-overrides\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.003583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002836 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-log-socket\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.003583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002848 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-etc-sysctl-conf\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.003583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002860 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-cni-netd\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.003583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002874 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/727b584c-bed9-4780-a327-b84b809591a9-iptables-alerter-script\") pod \"iptables-alerter-nxnxn\" (UID: \"727b584c-bed9-4780-a327-b84b809591a9\") " pod="openshift-network-operator/iptables-alerter-nxnxn" Jul 27 20:51:14.003583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002889 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-var-lib-kubelet\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.003583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002920 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-var-lib-openvswitch\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.004478 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002875 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db168b68-5746-4d6a-b2ee-fc30b50bf424-cni-binary-copy\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.004478 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002920 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:14.004478 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002954 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-host-var-lib-cni-multus\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.004478 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.002980 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-run-systemd\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.004478 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003004 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed75d226-bf8c-4148-8554-135bc455a794-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:14.004478 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003014 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-lib-modules\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.004478 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003027 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed75d226-bf8c-4148-8554-135bc455a794-etc-selinux\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:14.004478 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003043 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-run-systemd\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.004478 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003050 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4d712e4f-4317-4915-a4d0-2e02a6154d8c-ovnkube-config\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.004478 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003062 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78a12024-7c5c-4f1c-a15d-58876f88ee45-tmp-dir\") pod \"node-resolver-lbt9p\" (UID: \"78a12024-7c5c-4f1c-a15d-58876f88ee45\") " pod="openshift-dns/node-resolver-lbt9p" Jul 27 20:51:14.004478 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003076 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed75d226-bf8c-4148-8554-135bc455a794-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:14.004478 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003074 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/727b584c-bed9-4780-a327-b84b809591a9-host-slash\") pod \"iptables-alerter-nxnxn\" (UID: \"727b584c-bed9-4780-a327-b84b809591a9\") " pod="openshift-network-operator/iptables-alerter-nxnxn" Jul 27 20:51:14.004478 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003101 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-host\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.004478 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003107 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/727b584c-bed9-4780-a327-b84b809591a9-host-slash\") pod \"iptables-alerter-nxnxn\" (UID: \"727b584c-bed9-4780-a327-b84b809591a9\") " pod="openshift-network-operator/iptables-alerter-nxnxn" Jul 27 20:51:14.004478 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003115 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3705087b-83f8-4e75-84e2-ce01936e4657-host\") pod \"node-ca-74fh5\" (UID: \"3705087b-83f8-4e75-84e2-ce01936e4657\") " pod="openshift-image-registry/node-ca-74fh5" Jul 27 20:51:14.004478 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003071 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-etc-sysctl-conf\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.004478 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003132 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8a1339c-f7dc-4b4e-9a09-c82402afe736-konnectivity-ca\") pod \"konnectivity-agent-td7kp\" (UID: \"e8a1339c-f7dc-4b4e-9a09-c82402afe736\") " pod="kube-system/konnectivity-agent-td7kp" Jul 27 20:51:14.005307 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003149 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-cnibin\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.005307 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003157 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed75d226-bf8c-4148-8554-135bc455a794-etc-selinux\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:14.005307 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003164 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-multus-socket-dir-parent\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.005307 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003184 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed75d226-bf8c-4148-8554-135bc455a794-socket-dir\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:14.005307 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003199 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:14.005307 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003208 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3705087b-83f8-4e75-84e2-ce01936e4657-host\") pod \"node-ca-74fh5\" (UID: \"3705087b-83f8-4e75-84e2-ce01936e4657\") " pod="openshift-image-registry/node-ca-74fh5" Jul 27 20:51:14.005307 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003216 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-etc-sysconfig\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.005307 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003263 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-etc-kubernetes\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.005307 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003164 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-host\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.005307 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003289 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-run\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.005307 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003313 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3705087b-83f8-4e75-84e2-ce01936e4657-serviceca\") pod \"node-ca-74fh5\" (UID: \"3705087b-83f8-4e75-84e2-ce01936e4657\") " pod="openshift-image-registry/node-ca-74fh5" Jul 27 20:51:14.005307 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003338 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-system-cni-dir\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.005307 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003355 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4d712e4f-4317-4915-a4d0-2e02a6154d8c-env-overrides\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.005307 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003363 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed75d226-bf8c-4148-8554-135bc455a794-socket-dir\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:14.005307 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003362 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-host-var-lib-kubelet\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.005307 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003400 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed75d226-bf8c-4148-8554-135bc455a794-device-dir\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:14.005307 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003406 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-etc-kubernetes\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.005983 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003415 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4d712e4f-4317-4915-a4d0-2e02a6154d8c-ovnkube-config\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.005983 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003267 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-etc-sysconfig\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.005983 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003427 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8a1339c-f7dc-4b4e-9a09-c82402afe736-agent-certs\") pod \"konnectivity-agent-td7kp\" (UID: \"e8a1339c-f7dc-4b4e-9a09-c82402afe736\") " pod="kube-system/konnectivity-agent-td7kp" Jul 27 20:51:14.005983 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003460 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed75d226-bf8c-4148-8554-135bc455a794-device-dir\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:14.005983 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003548 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk2g6\" (UniqueName: \"kubernetes.io/projected/3705087b-83f8-4e75-84e2-ce01936e4657-kube-api-access-qk2g6\") pod \"node-ca-74fh5\" (UID: \"3705087b-83f8-4e75-84e2-ce01936e4657\") " pod="openshift-image-registry/node-ca-74fh5" Jul 27 20:51:14.005983 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003576 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-etc-kubernetes\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.005983 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003600 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-run-ovn-kubernetes\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.005983 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003643 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znkxb\" (UniqueName: \"kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb\") pod \"network-check-target-zt78l\" (UID: \"8add67af-ae5d-4d84-b040-7d8cc145e665\") " pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:14.005983 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003669 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5srj\" (UniqueName: \"kubernetes.io/projected/727b584c-bed9-4780-a327-b84b809591a9-kube-api-access-c5srj\") pod \"iptables-alerter-nxnxn\" (UID: \"727b584c-bed9-4780-a327-b84b809591a9\") " pod="openshift-network-operator/iptables-alerter-nxnxn" Jul 27 20:51:14.005983 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003690 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:14.005983 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003692 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 27 20:51:14.005983 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003772 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-run\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.005983 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003692 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-etc-sysctl-d\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.005983 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003789 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-etc-sysctl-d\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.005983 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003796 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-systemd-units\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.005983 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003805 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3705087b-83f8-4e75-84e2-ce01936e4657-serviceca\") pod \"node-ca-74fh5\" (UID: \"3705087b-83f8-4e75-84e2-ce01936e4657\") " pod="openshift-image-registry/node-ca-74fh5" Jul 27 20:51:14.005983 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003814 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-cni-binary-copy\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:14.006555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003832 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-os-release\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:14.006555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003836 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-run-ovn-kubernetes\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.006555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003855 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-dbus\") pod \"global-pull-secret-syncer-qzsr7\" (UID: \"86fb8fed-0e82-44c5-9c08-08a9b4d8eea3\") " pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:14.006555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003862 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-systemd-units\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.006555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003871 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-os-release\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:14.006555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003872 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4d712e4f-4317-4915-a4d0-2e02a6154d8c-ovn-node-metrics-cert\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.006555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004170 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw5xb\" (UniqueName: \"kubernetes.io/projected/ed75d226-bf8c-4148-8554-135bc455a794-kube-api-access-kw5xb\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:14.006555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004196 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-dbus\") pod \"global-pull-secret-syncer-qzsr7\" (UID: \"86fb8fed-0e82-44c5-9c08-08a9b4d8eea3\") " pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:14.006555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004224 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-kubelet-config\") pod \"global-pull-secret-syncer-qzsr7\" (UID: \"86fb8fed-0e82-44c5-9c08-08a9b4d8eea3\") " pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:14.006555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.003692 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e8a1339c-f7dc-4b4e-9a09-c82402afe736-konnectivity-ca\") pod \"konnectivity-agent-td7kp\" (UID: \"e8a1339c-f7dc-4b4e-9a09-c82402afe736\") " pod="kube-system/konnectivity-agent-td7kp" Jul 27 20:51:14.006555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004383 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-cni-binary-copy\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:14.006555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004433 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-run-ovn\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.006555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004450 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-kubelet-config\") pod \"global-pull-secret-syncer-qzsr7\" (UID: \"86fb8fed-0e82-44c5-9c08-08a9b4d8eea3\") " pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:14.006555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004492 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpkwr\" (UniqueName: \"kubernetes.io/projected/464f92c8-5626-4fef-9eff-137f0c713279-kube-api-access-jpkwr\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.006555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004516 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-run-ovn\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.006555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004519 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-multus-cni-dir\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.006555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004552 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-run-netns\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.007131 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004577 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4d712e4f-4317-4915-a4d0-2e02a6154d8c-ovnkube-script-lib\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.007131 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004602 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbdpl\" (UniqueName: \"kubernetes.io/projected/668f5b4e-867c-47c2-b868-cb3f79c14f2a-kube-api-access-qbdpl\") pod \"network-metrics-daemon-wk44n\" (UID: \"668f5b4e-867c-47c2-b868-cb3f79c14f2a\") " pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:14.007131 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004625 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-os-release\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.007131 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004614 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-run-netns\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.007131 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004647 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-host-run-k8s-cni-cncf-io\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.007131 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004668 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-host-run-netns\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.007131 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004690 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-cni-bin\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.007131 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004714 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs\") pod \"network-metrics-daemon-wk44n\" (UID: \"668f5b4e-867c-47c2-b868-cb3f79c14f2a\") " pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:14.007131 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004738 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/464f92c8-5626-4fef-9eff-137f0c713279-etc-tuned\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.007131 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004763 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-multus-conf-dir\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.007131 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:14.004860 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 20:51:14.007131 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:14.004913 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs podName:668f5b4e-867c-47c2-b868-cb3f79c14f2a nodeName:}" failed. No retries permitted until 2026-07-27 20:51:14.504898845 +0000 UTC m=+3.028016564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs") pod "network-metrics-daemon-wk44n" (UID: "668f5b4e-867c-47c2-b868-cb3f79c14f2a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 20:51:14.007131 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004874 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-host-run-multus-certs\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.007131 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004959 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-host-cni-bin\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.007131 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004973 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-node-log\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.007131 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.004995 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/464f92c8-5626-4fef-9eff-137f0c713279-tmp\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.007131 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.005046 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4d712e4f-4317-4915-a4d0-2e02a6154d8c-node-log\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.008002 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.005066 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4d712e4f-4317-4915-a4d0-2e02a6154d8c-ovnkube-script-lib\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.008002 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.005082 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-host-var-lib-cni-bin\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.008002 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.005125 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-system-cni-dir\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:14.008002 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.005160 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-etc-modprobe-d\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.008002 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.005185 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-etc-systemd\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.008002 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.005190 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-system-cni-dir\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:14.008002 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.005209 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db168b68-5746-4d6a-b2ee-fc30b50bf424-multus-daemon-config\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.008002 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.005251 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px2rp\" (UniqueName: \"kubernetes.io/projected/db168b68-5746-4d6a-b2ee-fc30b50bf424-kube-api-access-px2rp\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.008002 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.005288 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-etc-modprobe-d\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.008002 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.005332 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/464f92c8-5626-4fef-9eff-137f0c713279-etc-systemd\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.008002 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.007423 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/464f92c8-5626-4fef-9eff-137f0c713279-tmp\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.008002 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.007473 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4d712e4f-4317-4915-a4d0-2e02a6154d8c-ovn-node-metrics-cert\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.008002 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.007517 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e8a1339c-f7dc-4b4e-9a09-c82402afe736-agent-certs\") pod \"konnectivity-agent-td7kp\" (UID: \"e8a1339c-f7dc-4b4e-9a09-c82402afe736\") " pod="kube-system/konnectivity-agent-td7kp" Jul 27 20:51:14.008002 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.007817 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/464f92c8-5626-4fef-9eff-137f0c713279-etc-tuned\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.020071 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.016718 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sflw2\" (UniqueName: \"kubernetes.io/projected/78a12024-7c5c-4f1c-a15d-58876f88ee45-kube-api-access-sflw2\") pod \"node-resolver-lbt9p\" (UID: \"78a12024-7c5c-4f1c-a15d-58876f88ee45\") " pod="openshift-dns/node-resolver-lbt9p" Jul 27 20:51:14.020071 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:14.016970 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 20:51:14.020071 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:14.016993 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 20:51:14.020071 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:14.017007 2565 projected.go:196] Error preparing data for projected volume kube-api-access-znkxb for pod openshift-network-diagnostics/network-check-target-zt78l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 20:51:14.020071 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:14.017111 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb podName:8add67af-ae5d-4d84-b040-7d8cc145e665 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:14.517092293 +0000 UTC m=+3.040209993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-znkxb" (UniqueName: "kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb") pod "network-check-target-zt78l" (UID: "8add67af-ae5d-4d84-b040-7d8cc145e665") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 20:51:14.020071 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.018958 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5gzj\" (UniqueName: \"kubernetes.io/projected/4d712e4f-4317-4915-a4d0-2e02a6154d8c-kube-api-access-m5gzj\") pod \"ovnkube-node-9gbjx\" (UID: \"4d712e4f-4317-4915-a4d0-2e02a6154d8c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.023640 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.020122 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw5xb\" (UniqueName: \"kubernetes.io/projected/ed75d226-bf8c-4148-8554-135bc455a794-kube-api-access-kw5xb\") pod \"aws-ebs-csi-driver-node-c72lh\" (UID: \"ed75d226-bf8c-4148-8554-135bc455a794\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:14.023640 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.020266 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk2g6\" (UniqueName: \"kubernetes.io/projected/3705087b-83f8-4e75-84e2-ce01936e4657-kube-api-access-qk2g6\") pod \"node-ca-74fh5\" (UID: \"3705087b-83f8-4e75-84e2-ce01936e4657\") " pod="openshift-image-registry/node-ca-74fh5" Jul 27 20:51:14.023640 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.021949 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5srj\" (UniqueName: \"kubernetes.io/projected/727b584c-bed9-4780-a327-b84b809591a9-kube-api-access-c5srj\") pod \"iptables-alerter-nxnxn\" (UID: \"727b584c-bed9-4780-a327-b84b809591a9\") " pod="openshift-network-operator/iptables-alerter-nxnxn" Jul 27 20:51:14.023640 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.022814 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9ws7\" (UniqueName: \"kubernetes.io/projected/11c3cc6a-d95d-4c2c-9df9-c552aedd85bb-kube-api-access-r9ws7\") pod \"multus-additional-cni-plugins-h9lsk\" (UID: \"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb\") " pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:14.023640 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.022877 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbdpl\" (UniqueName: \"kubernetes.io/projected/668f5b4e-867c-47c2-b868-cb3f79c14f2a-kube-api-access-qbdpl\") pod \"network-metrics-daemon-wk44n\" (UID: \"668f5b4e-867c-47c2-b868-cb3f79c14f2a\") " pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:14.024655 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.024636 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpkwr\" (UniqueName: \"kubernetes.io/projected/464f92c8-5626-4fef-9eff-137f0c713279-kube-api-access-jpkwr\") pod \"tuned-njk5z\" (UID: \"464f92c8-5626-4fef-9eff-137f0c713279\") " pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.052161 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.052113 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-232.ec2.internal" event={"ID":"9233b835deeeffbb2950b8b4fe9f0d75","Type":"ContainerStarted","Data":"a9df9501ae70673fad04b812aef08ae35a041a892ca7617698290757e617803f"} Jul 27 20:51:14.053144 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.053116 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal" event={"ID":"8c2f8ede239810f1da80ef8aeebedd8a","Type":"ContainerStarted","Data":"a033f4fb947d57fd8da7678b28c643e4c9df021828bf2a77152013da24b8fd56"} Jul 27 20:51:14.106565 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106533 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-hostroot\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.106726 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106582 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db168b68-5746-4d6a-b2ee-fc30b50bf424-cni-binary-copy\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.106726 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106607 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-host-var-lib-cni-multus\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.106726 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106633 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-hostroot\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.106726 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106637 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-cnibin\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.106726 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106685 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-cnibin\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.106726 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106691 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-multus-socket-dir-parent\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.106726 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106726 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-system-cni-dir\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107090 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106736 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-host-var-lib-cni-multus\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107090 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106751 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-host-var-lib-kubelet\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107090 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106781 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-system-cni-dir\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107090 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106784 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-host-var-lib-kubelet\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107090 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106742 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-multus-socket-dir-parent\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107090 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106796 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-etc-kubernetes\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107090 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106849 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-multus-cni-dir\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107090 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106887 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-etc-kubernetes\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107090 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106906 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-os-release\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107090 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106913 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-multus-cni-dir\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107090 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106937 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-host-run-k8s-cni-cncf-io\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107090 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106961 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-host-run-netns\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107090 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106978 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-os-release\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107090 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106978 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-host-run-k8s-cni-cncf-io\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107090 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.106995 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-multus-conf-dir\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107090 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.107027 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-host-run-netns\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107090 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.107029 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-multus-conf-dir\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107090 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.107045 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-host-run-multus-certs\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107843 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.107065 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-host-var-lib-cni-bin\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107843 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.107075 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db168b68-5746-4d6a-b2ee-fc30b50bf424-cni-binary-copy\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107843 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.107082 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-host-run-multus-certs\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107843 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.107082 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db168b68-5746-4d6a-b2ee-fc30b50bf424-multus-daemon-config\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107843 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.107127 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px2rp\" (UniqueName: \"kubernetes.io/projected/db168b68-5746-4d6a-b2ee-fc30b50bf424-kube-api-access-px2rp\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107843 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.107123 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db168b68-5746-4d6a-b2ee-fc30b50bf424-host-var-lib-cni-bin\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.107843 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.107468 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db168b68-5746-4d6a-b2ee-fc30b50bf424-multus-daemon-config\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.117225 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.117205 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px2rp\" (UniqueName: \"kubernetes.io/projected/db168b68-5746-4d6a-b2ee-fc30b50bf424-kube-api-access-px2rp\") pod \"multus-6vdnr\" (UID: \"db168b68-5746-4d6a-b2ee-fc30b50bf424\") " pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.189689 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.189605 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lbt9p" Jul 27 20:51:14.200121 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.200098 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-74fh5" Jul 27 20:51:14.209181 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.209161 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" Jul 27 20:51:14.215230 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.215207 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h9lsk" Jul 27 20:51:14.222885 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.222864 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nxnxn" Jul 27 20:51:14.230533 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.230512 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-td7kp" Jul 27 20:51:14.237203 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.237184 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:14.245333 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.245315 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-njk5z" Jul 27 20:51:14.252147 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.252129 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6vdnr" Jul 27 20:51:14.510541 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.510460 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs\") pod \"network-metrics-daemon-wk44n\" (UID: \"668f5b4e-867c-47c2-b868-cb3f79c14f2a\") " pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:14.510541 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.510512 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret\") pod \"global-pull-secret-syncer-qzsr7\" (UID: \"86fb8fed-0e82-44c5-9c08-08a9b4d8eea3\") " pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:14.510772 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:14.510641 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 20:51:14.510772 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:14.510653 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 20:51:14.510772 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:14.510720 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs podName:668f5b4e-867c-47c2-b868-cb3f79c14f2a nodeName:}" failed. No retries permitted until 2026-07-27 20:51:15.510699681 +0000 UTC m=+4.033817380 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs") pod "network-metrics-daemon-wk44n" (UID: "668f5b4e-867c-47c2-b868-cb3f79c14f2a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 20:51:14.510772 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:14.510741 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret podName:86fb8fed-0e82-44c5-9c08-08a9b4d8eea3 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:15.510730556 +0000 UTC m=+4.033848257 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret") pod "global-pull-secret-syncer-qzsr7" (UID: "86fb8fed-0e82-44c5-9c08-08a9b4d8eea3") : object "kube-system"/"original-pull-secret" not registered Jul 27 20:51:14.611581 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.611549 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znkxb\" (UniqueName: \"kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb\") pod \"network-check-target-zt78l\" (UID: \"8add67af-ae5d-4d84-b040-7d8cc145e665\") " pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:14.611768 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:14.611691 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 20:51:14.611768 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:14.611707 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 20:51:14.611768 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:14.611720 2565 projected.go:196] Error preparing data for projected volume kube-api-access-znkxb for pod openshift-network-diagnostics/network-check-target-zt78l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 20:51:14.611768 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:14.611769 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb podName:8add67af-ae5d-4d84-b040-7d8cc145e665 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:15.611756729 +0000 UTC m=+4.134874445 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-znkxb" (UniqueName: "kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb") pod "network-check-target-zt78l" (UID: "8add67af-ae5d-4d84-b040-7d8cc145e665") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 20:51:14.938015 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.937978 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 20:46:13 +0000 UTC" deadline="2028-05-05 09:35:08.180864958 +0000 UTC" Jul 27 20:51:14.938625 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:14.938024 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15540h43m53.242859948s" Jul 27 20:51:15.007863 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:15.007820 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod727b584c_bed9_4780_a327_b84b809591a9.slice/crio-bae7eb4e7c987198e374836e67f2bc0bca1ebcca002af194b7cdcbb1ec2e0283 WatchSource:0}: Error finding container bae7eb4e7c987198e374836e67f2bc0bca1ebcca002af194b7cdcbb1ec2e0283: Status 404 returned error can't find the container with id bae7eb4e7c987198e374836e67f2bc0bca1ebcca002af194b7cdcbb1ec2e0283 Jul 27 20:51:15.010614 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:15.010564 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11c3cc6a_d95d_4c2c_9df9_c552aedd85bb.slice/crio-962d8c6dfff1322eec9a852ee65990d83d1e5637eccbf5d0ef95d455acaa63dd WatchSource:0}: Error finding container 962d8c6dfff1322eec9a852ee65990d83d1e5637eccbf5d0ef95d455acaa63dd: Status 404 returned error can't find the container with id 962d8c6dfff1322eec9a852ee65990d83d1e5637eccbf5d0ef95d455acaa63dd Jul 27 20:51:15.014352 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:15.013666 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8a1339c_f7dc_4b4e_9a09_c82402afe736.slice/crio-a74259399977fbc873f83ba12f47453a5c906313e7ce1ddfa7e870b0b0d02b68 WatchSource:0}: Error finding container a74259399977fbc873f83ba12f47453a5c906313e7ce1ddfa7e870b0b0d02b68: Status 404 returned error can't find the container with id a74259399977fbc873f83ba12f47453a5c906313e7ce1ddfa7e870b0b0d02b68 Jul 27 20:51:15.014352 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:15.014227 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d712e4f_4317_4915_a4d0_2e02a6154d8c.slice/crio-9f10e3b203294e4487c80383be166e86a22e1e7faf5c901c70e823126d7e86c7 WatchSource:0}: Error finding container 9f10e3b203294e4487c80383be166e86a22e1e7faf5c901c70e823126d7e86c7: Status 404 returned error can't find the container with id 9f10e3b203294e4487c80383be166e86a22e1e7faf5c901c70e823126d7e86c7 Jul 27 20:51:15.018678 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:15.018660 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded75d226_bf8c_4148_8554_135bc455a794.slice/crio-b76d4001b608c8159248b2610178f929e7d05b900e2bf6dab1bf46512e015dec WatchSource:0}: Error finding container b76d4001b608c8159248b2610178f929e7d05b900e2bf6dab1bf46512e015dec: Status 404 returned error can't find the container with id b76d4001b608c8159248b2610178f929e7d05b900e2bf6dab1bf46512e015dec Jul 27 20:51:15.020690 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:15.020100 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod464f92c8_5626_4fef_9eff_137f0c713279.slice/crio-52434931517dc930fcfb869566c480eae5427cefe794a1286e7fd88106985556 WatchSource:0}: Error finding container 52434931517dc930fcfb869566c480eae5427cefe794a1286e7fd88106985556: Status 404 returned error can't find the container with id 52434931517dc930fcfb869566c480eae5427cefe794a1286e7fd88106985556 Jul 27 20:51:15.022954 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:15.021475 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3705087b_83f8_4e75_84e2_ce01936e4657.slice/crio-bb87661464e9d0179a9e21f57acce440421fa7360c882e6ba6f232ddd8df8e56 WatchSource:0}: Error finding container bb87661464e9d0179a9e21f57acce440421fa7360c882e6ba6f232ddd8df8e56: Status 404 returned error can't find the container with id bb87661464e9d0179a9e21f57acce440421fa7360c882e6ba6f232ddd8df8e56 Jul 27 20:51:15.022954 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:15.022091 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb168b68_5746_4d6a_b2ee_fc30b50bf424.slice/crio-9d2bfadbd19e215d627130e7ed1ed16578a9cf35be63058adf52c503369278fb WatchSource:0}: Error finding container 9d2bfadbd19e215d627130e7ed1ed16578a9cf35be63058adf52c503369278fb: Status 404 returned error can't find the container with id 9d2bfadbd19e215d627130e7ed1ed16578a9cf35be63058adf52c503369278fb Jul 27 20:51:15.022954 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:15.022857 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78a12024_7c5c_4f1c_a15d_58876f88ee45.slice/crio-c5bb266207f2b1ae67ad3f57c6328e08b0cc0845c60958a9b6a7276d45e8231f WatchSource:0}: Error finding container c5bb266207f2b1ae67ad3f57c6328e08b0cc0845c60958a9b6a7276d45e8231f: Status 404 returned error can't find the container with id c5bb266207f2b1ae67ad3f57c6328e08b0cc0845c60958a9b6a7276d45e8231f Jul 27 20:51:15.047123 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:15.047105 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:15.047256 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:15.047220 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wk44n" podUID="668f5b4e-867c-47c2-b868-cb3f79c14f2a" Jul 27 20:51:15.058311 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:15.058107 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6vdnr" event={"ID":"db168b68-5746-4d6a-b2ee-fc30b50bf424","Type":"ContainerStarted","Data":"9d2bfadbd19e215d627130e7ed1ed16578a9cf35be63058adf52c503369278fb"} Jul 27 20:51:15.059790 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:15.059766 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-74fh5" event={"ID":"3705087b-83f8-4e75-84e2-ce01936e4657","Type":"ContainerStarted","Data":"bb87661464e9d0179a9e21f57acce440421fa7360c882e6ba6f232ddd8df8e56"} Jul 27 20:51:15.060755 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:15.060736 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lbt9p" event={"ID":"78a12024-7c5c-4f1c-a15d-58876f88ee45","Type":"ContainerStarted","Data":"c5bb266207f2b1ae67ad3f57c6328e08b0cc0845c60958a9b6a7276d45e8231f"} Jul 27 20:51:15.063422 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:15.063401 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" event={"ID":"ed75d226-bf8c-4148-8554-135bc455a794","Type":"ContainerStarted","Data":"b76d4001b608c8159248b2610178f929e7d05b900e2bf6dab1bf46512e015dec"} Jul 27 20:51:15.064652 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:15.064629 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9lsk" event={"ID":"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb","Type":"ContainerStarted","Data":"962d8c6dfff1322eec9a852ee65990d83d1e5637eccbf5d0ef95d455acaa63dd"} Jul 27 20:51:15.069569 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:15.069546 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nxnxn" event={"ID":"727b584c-bed9-4780-a327-b84b809591a9","Type":"ContainerStarted","Data":"bae7eb4e7c987198e374836e67f2bc0bca1ebcca002af194b7cdcbb1ec2e0283"} Jul 27 20:51:15.072491 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:15.072466 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-njk5z" event={"ID":"464f92c8-5626-4fef-9eff-137f0c713279","Type":"ContainerStarted","Data":"52434931517dc930fcfb869566c480eae5427cefe794a1286e7fd88106985556"} Jul 27 20:51:15.073906 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:15.073887 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" event={"ID":"4d712e4f-4317-4915-a4d0-2e02a6154d8c","Type":"ContainerStarted","Data":"9f10e3b203294e4487c80383be166e86a22e1e7faf5c901c70e823126d7e86c7"} Jul 27 20:51:15.074789 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:15.074768 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-td7kp" event={"ID":"e8a1339c-f7dc-4b4e-9a09-c82402afe736","Type":"ContainerStarted","Data":"a74259399977fbc873f83ba12f47453a5c906313e7ce1ddfa7e870b0b0d02b68"} Jul 27 20:51:15.517032 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:15.516948 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs\") pod \"network-metrics-daemon-wk44n\" (UID: \"668f5b4e-867c-47c2-b868-cb3f79c14f2a\") " pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:15.517032 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:15.517008 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret\") pod \"global-pull-secret-syncer-qzsr7\" (UID: \"86fb8fed-0e82-44c5-9c08-08a9b4d8eea3\") " pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:15.517280 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:15.517158 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 20:51:15.517280 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:15.517219 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret podName:86fb8fed-0e82-44c5-9c08-08a9b4d8eea3 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:17.517202632 +0000 UTC m=+6.040320345 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret") pod "global-pull-secret-syncer-qzsr7" (UID: "86fb8fed-0e82-44c5-9c08-08a9b4d8eea3") : object "kube-system"/"original-pull-secret" not registered Jul 27 20:51:15.517623 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:15.517604 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 20:51:15.517685 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:15.517658 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs podName:668f5b4e-867c-47c2-b868-cb3f79c14f2a nodeName:}" failed. No retries permitted until 2026-07-27 20:51:17.517642665 +0000 UTC m=+6.040760368 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs") pod "network-metrics-daemon-wk44n" (UID: "668f5b4e-867c-47c2-b868-cb3f79c14f2a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 20:51:15.617782 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:15.617742 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znkxb\" (UniqueName: \"kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb\") pod \"network-check-target-zt78l\" (UID: \"8add67af-ae5d-4d84-b040-7d8cc145e665\") " pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:15.617960 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:15.617939 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 20:51:15.618045 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:15.617967 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 20:51:15.618045 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:15.617980 2565 projected.go:196] Error preparing data for projected volume kube-api-access-znkxb for pod openshift-network-diagnostics/network-check-target-zt78l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 20:51:15.618142 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:15.618050 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb podName:8add67af-ae5d-4d84-b040-7d8cc145e665 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:17.618033032 +0000 UTC m=+6.141150736 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-znkxb" (UniqueName: "kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb") pod "network-check-target-zt78l" (UID: "8add67af-ae5d-4d84-b040-7d8cc145e665") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 20:51:15.939356 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:15.939201 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 20:46:13 +0000 UTC" deadline="2028-04-25 00:17:41.213285148 +0000 UTC" Jul 27 20:51:15.939356 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:15.939321 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15291h26m25.273968837s" Jul 27 20:51:16.049532 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:16.049212 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:16.049532 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:16.049365 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qzsr7" podUID="86fb8fed-0e82-44c5-9c08-08a9b4d8eea3" Jul 27 20:51:16.049532 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:16.049449 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:16.049532 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:16.049516 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zt78l" podUID="8add67af-ae5d-4d84-b040-7d8cc145e665" Jul 27 20:51:16.104562 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:16.103706 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-232.ec2.internal" event={"ID":"9233b835deeeffbb2950b8b4fe9f0d75","Type":"ContainerStarted","Data":"d24bc9578eda86a272f82b0fecc2301e0d5e1e384e635dd4dfc8a18d6bf0fea5"} Jul 27 20:51:16.121687 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:16.121652 2565 generic.go:356] "Generic (PLEG): container finished" podID="8c2f8ede239810f1da80ef8aeebedd8a" containerID="4d035d894378bc6f8d78785f66fdfd62ee3c7faa9d19a5728d215b371fe71c0a" exitCode=0 Jul 27 20:51:16.121889 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:16.121707 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal" event={"ID":"8c2f8ede239810f1da80ef8aeebedd8a","Type":"ContainerDied","Data":"4d035d894378bc6f8d78785f66fdfd62ee3c7faa9d19a5728d215b371fe71c0a"} Jul 27 20:51:16.172297 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:16.172178 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-232.ec2.internal" podStartSLOduration=3.172155907 podStartE2EDuration="3.172155907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:51:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:51:16.128329176 +0000 UTC m=+4.651446900" watchObservedRunningTime="2026-07-27 20:51:16.172155907 +0000 UTC m=+4.695273661" Jul 27 20:51:17.046774 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:17.046745 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:17.047218 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:17.046886 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wk44n" podUID="668f5b4e-867c-47c2-b868-cb3f79c14f2a" Jul 27 20:51:17.145361 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:17.145302 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal" event={"ID":"8c2f8ede239810f1da80ef8aeebedd8a","Type":"ContainerStarted","Data":"a13554e39557fa87c08581f5846ce35ccf76df08788939fc1d1a50540da39cb7"} Jul 27 20:51:17.535102 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:17.535059 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret\") pod \"global-pull-secret-syncer-qzsr7\" (UID: \"86fb8fed-0e82-44c5-9c08-08a9b4d8eea3\") " pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:17.535371 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:17.535175 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs\") pod \"network-metrics-daemon-wk44n\" (UID: \"668f5b4e-867c-47c2-b868-cb3f79c14f2a\") " pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:17.535371 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:17.535328 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 20:51:17.535496 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:17.535394 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs podName:668f5b4e-867c-47c2-b868-cb3f79c14f2a nodeName:}" failed. No retries permitted until 2026-07-27 20:51:21.535375652 +0000 UTC m=+10.058493357 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs") pod "network-metrics-daemon-wk44n" (UID: "668f5b4e-867c-47c2-b868-cb3f79c14f2a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 20:51:17.535838 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:17.535819 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 20:51:17.535983 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:17.535875 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret podName:86fb8fed-0e82-44c5-9c08-08a9b4d8eea3 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:21.535860552 +0000 UTC m=+10.058978254 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret") pod "global-pull-secret-syncer-qzsr7" (UID: "86fb8fed-0e82-44c5-9c08-08a9b4d8eea3") : object "kube-system"/"original-pull-secret" not registered Jul 27 20:51:17.635713 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:17.635669 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znkxb\" (UniqueName: \"kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb\") pod \"network-check-target-zt78l\" (UID: \"8add67af-ae5d-4d84-b040-7d8cc145e665\") " pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:17.635898 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:17.635879 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 20:51:17.635898 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:17.635899 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 20:51:17.636061 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:17.635911 2565 projected.go:196] Error preparing data for projected volume kube-api-access-znkxb for pod openshift-network-diagnostics/network-check-target-zt78l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 20:51:17.636061 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:17.635969 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb podName:8add67af-ae5d-4d84-b040-7d8cc145e665 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:21.635951178 +0000 UTC m=+10.159068885 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-znkxb" (UniqueName: "kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb") pod "network-check-target-zt78l" (UID: "8add67af-ae5d-4d84-b040-7d8cc145e665") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 20:51:18.048394 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:18.048358 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:18.048934 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:18.048498 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qzsr7" podUID="86fb8fed-0e82-44c5-9c08-08a9b4d8eea3" Jul 27 20:51:18.048934 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:18.048851 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:18.048934 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:18.048916 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zt78l" podUID="8add67af-ae5d-4d84-b040-7d8cc145e665" Jul 27 20:51:19.047109 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:19.046279 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:19.047109 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:19.046428 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wk44n" podUID="668f5b4e-867c-47c2-b868-cb3f79c14f2a" Jul 27 20:51:20.048896 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:20.048864 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:20.049280 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:20.048971 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qzsr7" podUID="86fb8fed-0e82-44c5-9c08-08a9b4d8eea3" Jul 27 20:51:20.049280 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:20.049272 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:20.049379 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:20.049328 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zt78l" podUID="8add67af-ae5d-4d84-b040-7d8cc145e665" Jul 27 20:51:21.046786 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:21.046735 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:21.046978 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:21.046898 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wk44n" podUID="668f5b4e-867c-47c2-b868-cb3f79c14f2a" Jul 27 20:51:21.569977 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:21.569939 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs\") pod \"network-metrics-daemon-wk44n\" (UID: \"668f5b4e-867c-47c2-b868-cb3f79c14f2a\") " pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:21.570440 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:21.569995 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret\") pod \"global-pull-secret-syncer-qzsr7\" (UID: \"86fb8fed-0e82-44c5-9c08-08a9b4d8eea3\") " pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:21.570440 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:21.570112 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 20:51:21.570440 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:21.570138 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 20:51:21.570440 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:21.570175 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret podName:86fb8fed-0e82-44c5-9c08-08a9b4d8eea3 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:29.570157915 +0000 UTC m=+18.093275631 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret") pod "global-pull-secret-syncer-qzsr7" (UID: "86fb8fed-0e82-44c5-9c08-08a9b4d8eea3") : object "kube-system"/"original-pull-secret" not registered Jul 27 20:51:21.570440 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:21.570220 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs podName:668f5b4e-867c-47c2-b868-cb3f79c14f2a nodeName:}" failed. No retries permitted until 2026-07-27 20:51:29.570207039 +0000 UTC m=+18.093324743 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs") pod "network-metrics-daemon-wk44n" (UID: "668f5b4e-867c-47c2-b868-cb3f79c14f2a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 20:51:21.671298 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:21.671263 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znkxb\" (UniqueName: \"kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb\") pod \"network-check-target-zt78l\" (UID: \"8add67af-ae5d-4d84-b040-7d8cc145e665\") " pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:21.671487 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:21.671457 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 20:51:21.671487 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:21.671489 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 20:51:21.671688 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:21.671503 2565 projected.go:196] Error preparing data for projected volume kube-api-access-znkxb for pod openshift-network-diagnostics/network-check-target-zt78l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 20:51:21.671688 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:21.671572 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb podName:8add67af-ae5d-4d84-b040-7d8cc145e665 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:29.671551883 +0000 UTC m=+18.194669604 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-znkxb" (UniqueName: "kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb") pod "network-check-target-zt78l" (UID: "8add67af-ae5d-4d84-b040-7d8cc145e665") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 20:51:22.048036 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:22.048001 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:22.048190 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:22.048123 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qzsr7" podUID="86fb8fed-0e82-44c5-9c08-08a9b4d8eea3" Jul 27 20:51:22.048190 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:22.048169 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:22.048300 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:22.048270 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zt78l" podUID="8add67af-ae5d-4d84-b040-7d8cc145e665" Jul 27 20:51:23.046853 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:23.046378 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:23.046853 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:23.046528 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wk44n" podUID="668f5b4e-867c-47c2-b868-cb3f79c14f2a" Jul 27 20:51:24.049504 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:24.047343 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:24.049504 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:24.047474 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zt78l" podUID="8add67af-ae5d-4d84-b040-7d8cc145e665" Jul 27 20:51:24.049504 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:24.047535 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:24.049504 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:24.047663 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qzsr7" podUID="86fb8fed-0e82-44c5-9c08-08a9b4d8eea3" Jul 27 20:51:25.046614 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:25.046575 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:25.046820 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:25.046721 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wk44n" podUID="668f5b4e-867c-47c2-b868-cb3f79c14f2a" Jul 27 20:51:26.047110 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:26.047065 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:26.047600 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:26.047066 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:26.047600 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:26.047228 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zt78l" podUID="8add67af-ae5d-4d84-b040-7d8cc145e665" Jul 27 20:51:26.047600 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:26.047403 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qzsr7" podUID="86fb8fed-0e82-44c5-9c08-08a9b4d8eea3" Jul 27 20:51:27.046690 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:27.046650 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:27.046861 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:27.046776 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wk44n" podUID="668f5b4e-867c-47c2-b868-cb3f79c14f2a" Jul 27 20:51:28.048957 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:28.048927 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:28.049337 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:28.049024 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zt78l" podUID="8add67af-ae5d-4d84-b040-7d8cc145e665" Jul 27 20:51:28.049337 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:28.049074 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:28.049337 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:28.049168 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qzsr7" podUID="86fb8fed-0e82-44c5-9c08-08a9b4d8eea3" Jul 27 20:51:29.046566 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:29.046528 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:29.046768 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:29.046681 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wk44n" podUID="668f5b4e-867c-47c2-b868-cb3f79c14f2a" Jul 27 20:51:29.635292 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:29.635258 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs\") pod \"network-metrics-daemon-wk44n\" (UID: \"668f5b4e-867c-47c2-b868-cb3f79c14f2a\") " pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:29.635684 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:29.635308 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret\") pod \"global-pull-secret-syncer-qzsr7\" (UID: \"86fb8fed-0e82-44c5-9c08-08a9b4d8eea3\") " pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:29.635684 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:29.635401 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 20:51:29.635684 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:29.635410 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 20:51:29.635684 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:29.635468 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs podName:668f5b4e-867c-47c2-b868-cb3f79c14f2a nodeName:}" failed. No retries permitted until 2026-07-27 20:51:45.635451921 +0000 UTC m=+34.158569621 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs") pod "network-metrics-daemon-wk44n" (UID: "668f5b4e-867c-47c2-b868-cb3f79c14f2a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 20:51:29.635684 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:29.635483 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret podName:86fb8fed-0e82-44c5-9c08-08a9b4d8eea3 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:45.635476688 +0000 UTC m=+34.158594387 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret") pod "global-pull-secret-syncer-qzsr7" (UID: "86fb8fed-0e82-44c5-9c08-08a9b4d8eea3") : object "kube-system"/"original-pull-secret" not registered Jul 27 20:51:29.736284 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:29.736171 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znkxb\" (UniqueName: \"kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb\") pod \"network-check-target-zt78l\" (UID: \"8add67af-ae5d-4d84-b040-7d8cc145e665\") " pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:29.736450 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:29.736392 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 20:51:29.736450 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:29.736418 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 20:51:29.736450 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:29.736431 2565 projected.go:196] Error preparing data for projected volume kube-api-access-znkxb for pod openshift-network-diagnostics/network-check-target-zt78l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 20:51:29.736603 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:29.736501 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb podName:8add67af-ae5d-4d84-b040-7d8cc145e665 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:45.73648226 +0000 UTC m=+34.259599982 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-znkxb" (UniqueName: "kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb") pod "network-check-target-zt78l" (UID: "8add67af-ae5d-4d84-b040-7d8cc145e665") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 20:51:30.051068 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:30.050568 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:30.051068 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:30.050693 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zt78l" podUID="8add67af-ae5d-4d84-b040-7d8cc145e665" Jul 27 20:51:30.051068 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:30.050743 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:30.051068 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:30.050863 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qzsr7" podUID="86fb8fed-0e82-44c5-9c08-08a9b4d8eea3" Jul 27 20:51:31.046661 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:31.046629 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:31.047099 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:31.046765 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wk44n" podUID="668f5b4e-867c-47c2-b868-cb3f79c14f2a" Jul 27 20:51:32.050010 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:32.049976 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:32.050446 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:32.050094 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qzsr7" podUID="86fb8fed-0e82-44c5-9c08-08a9b4d8eea3" Jul 27 20:51:32.050446 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:32.050150 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:32.050446 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:32.050275 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zt78l" podUID="8add67af-ae5d-4d84-b040-7d8cc145e665" Jul 27 20:51:33.047524 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:33.047017 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:33.047524 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:33.047151 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wk44n" podUID="668f5b4e-867c-47c2-b868-cb3f79c14f2a" Jul 27 20:51:34.047461 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:34.047201 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:34.047915 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:34.047258 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:34.047915 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:34.047589 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zt78l" podUID="8add67af-ae5d-4d84-b040-7d8cc145e665" Jul 27 20:51:34.047915 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:34.047604 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qzsr7" podUID="86fb8fed-0e82-44c5-9c08-08a9b4d8eea3" Jul 27 20:51:34.179608 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:34.179572 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6vdnr" event={"ID":"db168b68-5746-4d6a-b2ee-fc30b50bf424","Type":"ContainerStarted","Data":"39a89fb270b95f889ac8ab04a7e40172ebe1d04d80709eb375f88df1ae07abee"} Jul 27 20:51:34.180877 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:34.180855 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-74fh5" event={"ID":"3705087b-83f8-4e75-84e2-ce01936e4657","Type":"ContainerStarted","Data":"29056c421033a8bc5d5ee9c4abcfd1ed12737b9ecc65d64c402cef9fd11c3602"} Jul 27 20:51:34.182180 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:34.182159 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lbt9p" event={"ID":"78a12024-7c5c-4f1c-a15d-58876f88ee45","Type":"ContainerStarted","Data":"c92ad867641c282e8124258ac99e33c9268d8ef867dfb09dcf3b86b1d9ae2f8e"} Jul 27 20:51:34.183559 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:34.183539 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" event={"ID":"ed75d226-bf8c-4148-8554-135bc455a794","Type":"ContainerStarted","Data":"6f77a0130a69a449e7f5c3059330f05c41258b9dc8b2da4315241d9ed4d5fc32"} Jul 27 20:51:34.184989 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:34.184970 2565 generic.go:356] "Generic (PLEG): container finished" podID="11c3cc6a-d95d-4c2c-9df9-c552aedd85bb" containerID="17f824626c18535028437652b87e0c2a348da3e78112afe537dc488b9e288871" exitCode=0 Jul 27 20:51:34.185068 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:34.185020 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9lsk" event={"ID":"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb","Type":"ContainerDied","Data":"17f824626c18535028437652b87e0c2a348da3e78112afe537dc488b9e288871"} Jul 27 20:51:34.186478 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:34.186459 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-njk5z" event={"ID":"464f92c8-5626-4fef-9eff-137f0c713279","Type":"ContainerStarted","Data":"d19bd93fe1e236ca7da0c47904c767d3e1d507d3b3ac33c41ce947bf177e5dde"} Jul 27 20:51:34.187753 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:34.187732 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-td7kp" event={"ID":"e8a1339c-f7dc-4b4e-9a09-c82402afe736","Type":"ContainerStarted","Data":"adc5d0c397f60dcaddb6b66be2c9eda31f35d88c6e125b385e399c25450a9e3e"} Jul 27 20:51:34.198698 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:34.198658 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-232.ec2.internal" podStartSLOduration=21.198635561 podStartE2EDuration="21.198635561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:51:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:51:17.164775857 +0000 UTC m=+5.687893581" watchObservedRunningTime="2026-07-27 20:51:34.198635561 +0000 UTC m=+22.721753284" Jul 27 20:51:34.198985 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:34.198964 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6vdnr" podStartSLOduration=3.436994928 podStartE2EDuration="22.198958822s" totalImagesPullingTime="18.761963894s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:51:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:51:34.198434319 +0000 UTC m=+22.721552043" watchObservedRunningTime="2026-07-27 20:51:34.198958822 +0000 UTC m=+22.722076543" Jul 27 20:51:34.215450 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:34.215407 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-74fh5" podStartSLOduration=3.607175967 podStartE2EDuration="22.215393322s" totalImagesPullingTime="18.608217355s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:51:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:51:34.215271816 +0000 UTC m=+22.738389543" watchObservedRunningTime="2026-07-27 20:51:34.215393322 +0000 UTC m=+22.738511043" Jul 27 20:51:34.230518 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:34.230469 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-td7kp" podStartSLOduration=7.609377326 podStartE2EDuration="22.2304548s" totalImagesPullingTime="14.621077474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:51:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:51:34.230315667 +0000 UTC m=+22.753433389" watchObservedRunningTime="2026-07-27 20:51:34.2304548 +0000 UTC m=+22.753572521" Jul 27 20:51:34.271467 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:34.271406 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-njk5z" podStartSLOduration=3.65995855 podStartE2EDuration="22.271388483s" totalImagesPullingTime="18.611429933s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:51:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:51:34.249411828 +0000 UTC m=+22.772529550" watchObservedRunningTime="2026-07-27 20:51:34.271388483 +0000 UTC m=+22.794506205" Jul 27 20:51:34.271915 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:34.271877 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lbt9p" podStartSLOduration=3.663939145 podStartE2EDuration="22.271866318s" totalImagesPullingTime="18.607927173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:51:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:51:34.271350327 +0000 UTC m=+22.794468050" watchObservedRunningTime="2026-07-27 20:51:34.271866318 +0000 UTC m=+22.794984043" Jul 27 20:51:34.999402 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:34.999218 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 27 20:51:35.047102 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:35.047072 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:35.047280 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:35.047221 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wk44n" podUID="668f5b4e-867c-47c2-b868-cb3f79c14f2a" Jul 27 20:51:35.191841 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:35.191804 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nxnxn" event={"ID":"727b584c-bed9-4780-a327-b84b809591a9","Type":"ContainerStarted","Data":"2f75446c4bd96331de80aec226eeb6e07977eba60c5fbf1a6acd280a7cce4d66"} Jul 27 20:51:35.199481 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:35.199452 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" event={"ID":"4d712e4f-4317-4915-a4d0-2e02a6154d8c","Type":"ContainerStarted","Data":"74d80078708ffb363cd4d6aa268065c4db84fdccf523b667b63d3eab9e252098"} Jul 27 20:51:35.199610 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:35.199538 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" event={"ID":"4d712e4f-4317-4915-a4d0-2e02a6154d8c","Type":"ContainerStarted","Data":"de280fe3b5d530f243445d10902088954123f1e57431799e18bb32769c22fd84"} Jul 27 20:51:35.199610 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:35.199598 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" event={"ID":"4d712e4f-4317-4915-a4d0-2e02a6154d8c","Type":"ContainerStarted","Data":"c7b213886d7320c2f7a9fb961c7879e9f7595d553c769276de914906e4d46461"} Jul 27 20:51:35.201407 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:35.201380 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" event={"ID":"ed75d226-bf8c-4148-8554-135bc455a794","Type":"ContainerStarted","Data":"75613dce6e2e2ea046c02b582e2647fb1cd205d4df5c571eee64a3f7457c050a"} Jul 27 20:51:35.210725 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:35.210682 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nxnxn" podStartSLOduration=4.58802941 podStartE2EDuration="23.210665924s" totalImagesPullingTime="18.622636514s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:51:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:51:35.210329059 +0000 UTC m=+23.733446784" watchObservedRunningTime="2026-07-27 20:51:35.210665924 +0000 UTC m=+23.733783647" Jul 27 20:51:35.962206 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:35.962081 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-27T20:51:34.999395409Z","UUID":"e7966c7e-ca0d-4cbc-9598-32a7e2c69518","Handler":null,"Name":"","Endpoint":""} Jul 27 20:51:35.963641 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:35.963621 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 27 20:51:35.963893 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:35.963658 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 27 20:51:36.047284 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:36.047251 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:36.047557 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:36.047394 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zt78l" podUID="8add67af-ae5d-4d84-b040-7d8cc145e665" Jul 27 20:51:36.047557 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:36.047461 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:36.047687 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:36.047583 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qzsr7" podUID="86fb8fed-0e82-44c5-9c08-08a9b4d8eea3" Jul 27 20:51:36.207518 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:36.207484 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" event={"ID":"4d712e4f-4317-4915-a4d0-2e02a6154d8c","Type":"ContainerStarted","Data":"c7e216b4dc5e3af1c33e2b9ee397c9e463ca2a9b7c2aa4509a7d35b0ac851431"} Jul 27 20:51:36.208109 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:36.207570 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" event={"ID":"4d712e4f-4317-4915-a4d0-2e02a6154d8c","Type":"ContainerStarted","Data":"af00691aa422d7edba9c51ffab3282930cae95c36f901a4aba4425465aa90568"} Jul 27 20:51:36.208109 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:36.207602 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" event={"ID":"4d712e4f-4317-4915-a4d0-2e02a6154d8c","Type":"ContainerStarted","Data":"8791521e893f90c8d091a084af45e90d6508c03baad44aa8a00c14be97d08ba3"} Jul 27 20:51:36.857978 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:36.857944 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-td7kp" Jul 27 20:51:36.858731 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:36.858705 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-td7kp" Jul 27 20:51:37.046583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:37.046546 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:37.046751 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:37.046684 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wk44n" podUID="668f5b4e-867c-47c2-b868-cb3f79c14f2a" Jul 27 20:51:37.212533 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:37.212326 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" event={"ID":"ed75d226-bf8c-4148-8554-135bc455a794","Type":"ContainerStarted","Data":"faeb561f86fdcf4a1ab43633a09757d8466981276552e0839eb34c2da47cc104"} Jul 27 20:51:37.213006 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:37.212572 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-td7kp" Jul 27 20:51:37.213417 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:37.213386 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-td7kp" Jul 27 20:51:37.234090 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:37.234030 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c72lh" podStartSLOduration=4.457535751 podStartE2EDuration="25.234010948s" totalImagesPullingTime="20.776475197s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:51:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:51:37.232984573 +0000 UTC m=+25.756102296" watchObservedRunningTime="2026-07-27 20:51:37.234010948 +0000 UTC m=+25.757128672" Jul 27 20:51:38.046438 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:38.046405 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:38.046610 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:38.046520 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qzsr7" podUID="86fb8fed-0e82-44c5-9c08-08a9b4d8eea3" Jul 27 20:51:38.046610 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:38.046585 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:38.046730 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:38.046693 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zt78l" podUID="8add67af-ae5d-4d84-b040-7d8cc145e665" Jul 27 20:51:39.046172 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:39.046149 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:39.046762 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:39.046280 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wk44n" podUID="668f5b4e-867c-47c2-b868-cb3f79c14f2a" Jul 27 20:51:39.220348 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:39.220256 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" event={"ID":"4d712e4f-4317-4915-a4d0-2e02a6154d8c","Type":"ContainerStarted","Data":"8448d0b97568d3ed798178d18ba520a61a0e3277cf407af6dd074dcb7cb77440"} Jul 27 20:51:39.221910 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:39.221887 2565 generic.go:356] "Generic (PLEG): container finished" podID="11c3cc6a-d95d-4c2c-9df9-c552aedd85bb" containerID="4b520d066685078c3eccaf796dd271fa28827abb95113cb91b093a8d9fde7f31" exitCode=0 Jul 27 20:51:39.222038 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:39.221965 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9lsk" event={"ID":"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb","Type":"ContainerDied","Data":"4b520d066685078c3eccaf796dd271fa28827abb95113cb91b093a8d9fde7f31"} Jul 27 20:51:40.047085 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:40.047056 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:40.047471 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:40.047095 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:40.047471 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:40.047170 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qzsr7" podUID="86fb8fed-0e82-44c5-9c08-08a9b4d8eea3" Jul 27 20:51:40.047471 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:40.047297 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zt78l" podUID="8add67af-ae5d-4d84-b040-7d8cc145e665" Jul 27 20:51:41.046852 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:41.046581 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:41.047036 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:41.046855 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wk44n" podUID="668f5b4e-867c-47c2-b868-cb3f79c14f2a" Jul 27 20:51:41.230008 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:41.229973 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" event={"ID":"4d712e4f-4317-4915-a4d0-2e02a6154d8c","Type":"ContainerStarted","Data":"6f6fae9067f38fea87e5638c005feccb5aa6ee73be9a19c9c965d9cdb692196d"} Jul 27 20:51:41.230593 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:41.230202 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:41.232174 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:41.232153 2565 generic.go:356] "Generic (PLEG): container finished" podID="11c3cc6a-d95d-4c2c-9df9-c552aedd85bb" containerID="6980a2a4cd00eef8f81e3f8a779a64c1d191c16a914530e72662b34d13eac485" exitCode=0 Jul 27 20:51:41.232285 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:41.232184 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9lsk" event={"ID":"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb","Type":"ContainerDied","Data":"6980a2a4cd00eef8f81e3f8a779a64c1d191c16a914530e72662b34d13eac485"} Jul 27 20:51:41.247629 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:41.247608 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:41.262008 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:41.261958 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" podStartSLOduration=9.675245524 podStartE2EDuration="29.261945475s" totalImagesPullingTime="19.586699951s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:51:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:51:41.26120538 +0000 UTC m=+29.784323101" watchObservedRunningTime="2026-07-27 20:51:41.261945475 +0000 UTC m=+29.785063194" Jul 27 20:51:42.047628 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:42.047196 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:42.047628 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:42.047292 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:42.047628 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:42.047312 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zt78l" podUID="8add67af-ae5d-4d84-b040-7d8cc145e665" Jul 27 20:51:42.047628 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:42.047403 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qzsr7" podUID="86fb8fed-0e82-44c5-9c08-08a9b4d8eea3" Jul 27 20:51:42.234954 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:42.234925 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:42.235474 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:42.235005 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:42.251016 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:42.250993 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:51:43.046805 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:43.046776 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:43.046940 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:43.046874 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wk44n" podUID="668f5b4e-867c-47c2-b868-cb3f79c14f2a" Jul 27 20:51:43.103583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:43.103557 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qzsr7"] Jul 27 20:51:43.103710 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:43.103675 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:43.103788 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:43.103766 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qzsr7" podUID="86fb8fed-0e82-44c5-9c08-08a9b4d8eea3" Jul 27 20:51:43.107860 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:43.107835 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wk44n"] Jul 27 20:51:43.121074 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:43.121051 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zt78l"] Jul 27 20:51:43.121210 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:43.121146 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:43.121282 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:43.121216 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zt78l" podUID="8add67af-ae5d-4d84-b040-7d8cc145e665" Jul 27 20:51:43.238883 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:43.238698 2565 generic.go:356] "Generic (PLEG): container finished" podID="11c3cc6a-d95d-4c2c-9df9-c552aedd85bb" containerID="cc7986061de1ee2d15a37f9c63f009fe5a974f1bd4405eac5942b22a15d1904a" exitCode=0 Jul 27 20:51:43.239269 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:43.238765 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9lsk" event={"ID":"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb","Type":"ContainerDied","Data":"cc7986061de1ee2d15a37f9c63f009fe5a974f1bd4405eac5942b22a15d1904a"} Jul 27 20:51:43.239269 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:43.238941 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:43.239269 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:43.239038 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wk44n" podUID="668f5b4e-867c-47c2-b868-cb3f79c14f2a" Jul 27 20:51:45.047062 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:45.047027 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:45.047759 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:45.047069 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:45.047759 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:45.047145 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:45.047759 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:45.047163 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qzsr7" podUID="86fb8fed-0e82-44c5-9c08-08a9b4d8eea3" Jul 27 20:51:45.047759 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:45.047280 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wk44n" podUID="668f5b4e-867c-47c2-b868-cb3f79c14f2a" Jul 27 20:51:45.047759 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:45.047365 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zt78l" podUID="8add67af-ae5d-4d84-b040-7d8cc145e665" Jul 27 20:51:45.656008 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:45.655961 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret\") pod \"global-pull-secret-syncer-qzsr7\" (UID: \"86fb8fed-0e82-44c5-9c08-08a9b4d8eea3\") " pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:45.656174 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:45.656070 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs\") pod \"network-metrics-daemon-wk44n\" (UID: \"668f5b4e-867c-47c2-b868-cb3f79c14f2a\") " pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:45.656174 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:45.656130 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 20:51:45.656174 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:45.656167 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 20:51:45.656364 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:45.656206 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret podName:86fb8fed-0e82-44c5-9c08-08a9b4d8eea3 nodeName:}" failed. No retries permitted until 2026-07-27 20:52:17.656184571 +0000 UTC m=+66.179302293 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret") pod "global-pull-secret-syncer-qzsr7" (UID: "86fb8fed-0e82-44c5-9c08-08a9b4d8eea3") : object "kube-system"/"original-pull-secret" not registered Jul 27 20:51:45.656364 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:45.656226 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs podName:668f5b4e-867c-47c2-b868-cb3f79c14f2a nodeName:}" failed. No retries permitted until 2026-07-27 20:52:17.656217989 +0000 UTC m=+66.179335690 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs") pod "network-metrics-daemon-wk44n" (UID: "668f5b4e-867c-47c2-b868-cb3f79c14f2a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 20:51:45.757379 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:45.757344 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znkxb\" (UniqueName: \"kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb\") pod \"network-check-target-zt78l\" (UID: \"8add67af-ae5d-4d84-b040-7d8cc145e665\") " pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:45.757543 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:45.757530 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 20:51:45.757596 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:45.757552 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 20:51:45.757596 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:45.757562 2565 projected.go:196] Error preparing data for projected volume kube-api-access-znkxb for pod openshift-network-diagnostics/network-check-target-zt78l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 20:51:45.757711 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:45.757615 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb podName:8add67af-ae5d-4d84-b040-7d8cc145e665 nodeName:}" failed. No retries permitted until 2026-07-27 20:52:17.757598996 +0000 UTC m=+66.280716727 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-znkxb" (UniqueName: "kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb") pod "network-check-target-zt78l" (UID: "8add67af-ae5d-4d84-b040-7d8cc145e665") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 20:51:47.046548 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.046513 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:47.047032 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.046560 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:47.047032 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.046630 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:47.047032 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:47.046641 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qzsr7" podUID="86fb8fed-0e82-44c5-9c08-08a9b4d8eea3" Jul 27 20:51:47.047032 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:47.046698 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zt78l" podUID="8add67af-ae5d-4d84-b040-7d8cc145e665" Jul 27 20:51:47.047032 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:47.046778 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wk44n" podUID="668f5b4e-867c-47c2-b868-cb3f79c14f2a" Jul 27 20:51:47.251063 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.251024 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9lsk" event={"ID":"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb","Type":"ContainerStarted","Data":"f5dcf871b505000b58323c0db95527cebab55a2c12cc20b7e592cbb0445fd6fd"} Jul 27 20:51:47.780849 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.780824 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-232.ec2.internal" event="NodeReady" Jul 27 20:51:47.781011 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.780947 2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 27 20:51:47.822147 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.822116 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7655cf7f49-ldmrw"] Jul 27 20:51:47.836364 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.836326 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:47.837452 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.837428 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-j2k9d"] Jul 27 20:51:47.839285 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.839266 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 27 20:51:47.839398 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.839292 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 27 20:51:47.840031 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.840013 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xqs7p\"" Jul 27 20:51:47.840159 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.840051 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 27 20:51:47.849251 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.849220 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 27 20:51:47.850288 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.850272 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7655cf7f49-ldmrw"] Jul 27 20:51:47.850336 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.850300 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5psdr"] Jul 27 20:51:47.850392 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.850380 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j2k9d" Jul 27 20:51:47.852692 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.852673 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2qj4g\"" Jul 27 20:51:47.852805 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.852788 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 27 20:51:47.852923 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.852908 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 27 20:51:47.870288 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.870262 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-j2k9d"] Jul 27 20:51:47.870374 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.870298 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5psdr"] Jul 27 20:51:47.870374 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.870312 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5psdr" Jul 27 20:51:47.872798 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.872781 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cgmf5\"" Jul 27 20:51:47.872915 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.872899 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 27 20:51:47.872981 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.872926 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 27 20:51:47.873209 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.873191 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 27 20:51:47.972581 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.972550 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/43cb7481-cfd4-4480-8523-3b2907b10931-image-registry-private-configuration\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:47.972722 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.972585 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-bound-sa-token\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:47.972722 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.972610 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:47.972722 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.972632 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/43cb7481-cfd4-4480-8523-3b2907b10931-trusted-ca\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:47.972722 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.972669 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4cxf\" (UniqueName: \"kubernetes.io/projected/78365e64-dcf0-450f-ae1e-d8dc161e3657-kube-api-access-g4cxf\") pod \"ingress-canary-5psdr\" (UID: \"78365e64-dcf0-450f-ae1e-d8dc161e3657\") " pod="openshift-ingress-canary/ingress-canary-5psdr" Jul 27 20:51:47.972722 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.972692 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/43cb7481-cfd4-4480-8523-3b2907b10931-installation-pull-secrets\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:47.972722 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.972714 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwzk9\" (UniqueName: \"kubernetes.io/projected/c1f55707-7360-492b-a75f-f2211a512fd3-kube-api-access-dwzk9\") pod \"dns-default-j2k9d\" (UID: \"c1f55707-7360-492b-a75f-f2211a512fd3\") " pod="openshift-dns/dns-default-j2k9d" Jul 27 20:51:47.972949 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.972733 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/43cb7481-cfd4-4480-8523-3b2907b10931-ca-trust-extracted\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:47.972949 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.972746 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78365e64-dcf0-450f-ae1e-d8dc161e3657-cert\") pod \"ingress-canary-5psdr\" (UID: \"78365e64-dcf0-450f-ae1e-d8dc161e3657\") " pod="openshift-ingress-canary/ingress-canary-5psdr" Jul 27 20:51:47.972949 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.972762 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls\") pod \"dns-default-j2k9d\" (UID: \"c1f55707-7360-492b-a75f-f2211a512fd3\") " pod="openshift-dns/dns-default-j2k9d" Jul 27 20:51:47.972949 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.972807 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1f55707-7360-492b-a75f-f2211a512fd3-tmp-dir\") pod \"dns-default-j2k9d\" (UID: \"c1f55707-7360-492b-a75f-f2211a512fd3\") " pod="openshift-dns/dns-default-j2k9d" Jul 27 20:51:47.972949 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.972849 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/43cb7481-cfd4-4480-8523-3b2907b10931-registry-certificates\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:47.972949 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.972866 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc589\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-kube-api-access-vc589\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:47.972949 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:47.972882 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c1f55707-7360-492b-a75f-f2211a512fd3-config-volume\") pod \"dns-default-j2k9d\" (UID: \"c1f55707-7360-492b-a75f-f2211a512fd3\") " pod="openshift-dns/dns-default-j2k9d" Jul 27 20:51:48.073705 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.073641 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-bound-sa-token\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:48.073705 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.073676 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:48.073705 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.073693 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/43cb7481-cfd4-4480-8523-3b2907b10931-trusted-ca\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:48.074378 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:48.073794 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 20:51:48.074378 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:48.073814 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7655cf7f49-ldmrw: secret "image-registry-tls" not found Jul 27 20:51:48.074378 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:48.073879 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls podName:43cb7481-cfd4-4480-8523-3b2907b10931 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:48.573859672 +0000 UTC m=+37.096977372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls") pod "image-registry-7655cf7f49-ldmrw" (UID: "43cb7481-cfd4-4480-8523-3b2907b10931") : secret "image-registry-tls" not found Jul 27 20:51:48.074378 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.073934 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4cxf\" (UniqueName: \"kubernetes.io/projected/78365e64-dcf0-450f-ae1e-d8dc161e3657-kube-api-access-g4cxf\") pod \"ingress-canary-5psdr\" (UID: \"78365e64-dcf0-450f-ae1e-d8dc161e3657\") " pod="openshift-ingress-canary/ingress-canary-5psdr" Jul 27 20:51:48.074378 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.073979 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/43cb7481-cfd4-4480-8523-3b2907b10931-installation-pull-secrets\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:48.074378 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.074004 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwzk9\" (UniqueName: \"kubernetes.io/projected/c1f55707-7360-492b-a75f-f2211a512fd3-kube-api-access-dwzk9\") pod \"dns-default-j2k9d\" (UID: \"c1f55707-7360-492b-a75f-f2211a512fd3\") " pod="openshift-dns/dns-default-j2k9d" Jul 27 20:51:48.074378 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.074043 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/43cb7481-cfd4-4480-8523-3b2907b10931-ca-trust-extracted\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:48.074378 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.074067 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78365e64-dcf0-450f-ae1e-d8dc161e3657-cert\") pod \"ingress-canary-5psdr\" (UID: \"78365e64-dcf0-450f-ae1e-d8dc161e3657\") " pod="openshift-ingress-canary/ingress-canary-5psdr" Jul 27 20:51:48.074378 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.074103 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls\") pod \"dns-default-j2k9d\" (UID: \"c1f55707-7360-492b-a75f-f2211a512fd3\") " pod="openshift-dns/dns-default-j2k9d" Jul 27 20:51:48.074378 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.074135 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1f55707-7360-492b-a75f-f2211a512fd3-tmp-dir\") pod \"dns-default-j2k9d\" (UID: \"c1f55707-7360-492b-a75f-f2211a512fd3\") " pod="openshift-dns/dns-default-j2k9d" Jul 27 20:51:48.074378 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.074181 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/43cb7481-cfd4-4480-8523-3b2907b10931-registry-certificates\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:48.074378 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.074214 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc589\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-kube-api-access-vc589\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:48.074378 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:48.074221 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 20:51:48.074378 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.074266 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c1f55707-7360-492b-a75f-f2211a512fd3-config-volume\") pod \"dns-default-j2k9d\" (UID: \"c1f55707-7360-492b-a75f-f2211a512fd3\") " pod="openshift-dns/dns-default-j2k9d" Jul 27 20:51:48.074378 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:48.074310 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls podName:c1f55707-7360-492b-a75f-f2211a512fd3 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:48.574291531 +0000 UTC m=+37.097409233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls") pod "dns-default-j2k9d" (UID: "c1f55707-7360-492b-a75f-f2211a512fd3") : secret "dns-default-metrics-tls" not found Jul 27 20:51:48.074378 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.074347 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/43cb7481-cfd4-4480-8523-3b2907b10931-image-registry-private-configuration\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:48.075091 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.074431 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/43cb7481-cfd4-4480-8523-3b2907b10931-ca-trust-extracted\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:48.075091 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.074759 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/43cb7481-cfd4-4480-8523-3b2907b10931-trusted-ca\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:48.075091 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.074758 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/43cb7481-cfd4-4480-8523-3b2907b10931-registry-certificates\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:48.075091 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.074807 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c1f55707-7360-492b-a75f-f2211a512fd3-config-volume\") pod \"dns-default-j2k9d\" (UID: \"c1f55707-7360-492b-a75f-f2211a512fd3\") " pod="openshift-dns/dns-default-j2k9d" Jul 27 20:51:48.075091 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.074887 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1f55707-7360-492b-a75f-f2211a512fd3-tmp-dir\") pod \"dns-default-j2k9d\" (UID: \"c1f55707-7360-492b-a75f-f2211a512fd3\") " pod="openshift-dns/dns-default-j2k9d" Jul 27 20:51:48.078191 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.078168 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/43cb7481-cfd4-4480-8523-3b2907b10931-installation-pull-secrets\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:48.078191 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.078183 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78365e64-dcf0-450f-ae1e-d8dc161e3657-cert\") pod \"ingress-canary-5psdr\" (UID: \"78365e64-dcf0-450f-ae1e-d8dc161e3657\") " pod="openshift-ingress-canary/ingress-canary-5psdr" Jul 27 20:51:48.078350 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.078168 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/43cb7481-cfd4-4480-8523-3b2907b10931-image-registry-private-configuration\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:48.084281 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.084258 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-bound-sa-token\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:48.084470 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.084448 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4cxf\" (UniqueName: \"kubernetes.io/projected/78365e64-dcf0-450f-ae1e-d8dc161e3657-kube-api-access-g4cxf\") pod \"ingress-canary-5psdr\" (UID: \"78365e64-dcf0-450f-ae1e-d8dc161e3657\") " pod="openshift-ingress-canary/ingress-canary-5psdr" Jul 27 20:51:48.085151 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.085129 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwzk9\" (UniqueName: \"kubernetes.io/projected/c1f55707-7360-492b-a75f-f2211a512fd3-kube-api-access-dwzk9\") pod \"dns-default-j2k9d\" (UID: \"c1f55707-7360-492b-a75f-f2211a512fd3\") " pod="openshift-dns/dns-default-j2k9d" Jul 27 20:51:48.085348 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.085328 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc589\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-kube-api-access-vc589\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:48.178099 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.178067 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5psdr" Jul 27 20:51:48.256480 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.256318 2565 generic.go:356] "Generic (PLEG): container finished" podID="11c3cc6a-d95d-4c2c-9df9-c552aedd85bb" containerID="f5dcf871b505000b58323c0db95527cebab55a2c12cc20b7e592cbb0445fd6fd" exitCode=0 Jul 27 20:51:48.256601 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.256403 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9lsk" event={"ID":"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb","Type":"ContainerDied","Data":"f5dcf871b505000b58323c0db95527cebab55a2c12cc20b7e592cbb0445fd6fd"} Jul 27 20:51:48.382889 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.382856 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5psdr"] Jul 27 20:51:48.385931 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:48.385906 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78365e64_dcf0_450f_ae1e_d8dc161e3657.slice/crio-b338a7699f27bba5d8e39f41ddaf87d8b8a7eb6eff03ef0d171b91987e93950e WatchSource:0}: Error finding container b338a7699f27bba5d8e39f41ddaf87d8b8a7eb6eff03ef0d171b91987e93950e: Status 404 returned error can't find the container with id b338a7699f27bba5d8e39f41ddaf87d8b8a7eb6eff03ef0d171b91987e93950e Jul 27 20:51:48.578371 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.578337 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:48.578538 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:48.578403 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls\") pod \"dns-default-j2k9d\" (UID: \"c1f55707-7360-492b-a75f-f2211a512fd3\") " pod="openshift-dns/dns-default-j2k9d" Jul 27 20:51:48.578538 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:48.578487 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 20:51:48.578613 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:48.578553 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls podName:c1f55707-7360-492b-a75f-f2211a512fd3 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:49.578534984 +0000 UTC m=+38.101652683 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls") pod "dns-default-j2k9d" (UID: "c1f55707-7360-492b-a75f-f2211a512fd3") : secret "dns-default-metrics-tls" not found Jul 27 20:51:48.578613 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:48.578490 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 20:51:48.578613 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:48.578591 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7655cf7f49-ldmrw: secret "image-registry-tls" not found Jul 27 20:51:48.578729 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:48.578653 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls podName:43cb7481-cfd4-4480-8523-3b2907b10931 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:49.578630342 +0000 UTC m=+38.101748043 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls") pod "image-registry-7655cf7f49-ldmrw" (UID: "43cb7481-cfd4-4480-8523-3b2907b10931") : secret "image-registry-tls" not found Jul 27 20:51:49.046574 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:49.046533 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:51:49.046764 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:49.046660 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:51:49.046764 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:49.046699 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:51:49.049609 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:49.049585 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 27 20:51:49.049739 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:49.049636 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2w5wq\"" Jul 27 20:51:49.049739 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:49.049712 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 27 20:51:49.049739 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:49.049728 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 27 20:51:49.049896 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:49.049756 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6wczd\"" Jul 27 20:51:49.050055 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:49.050039 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 27 20:51:49.262873 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:49.262839 2565 generic.go:356] "Generic (PLEG): container finished" podID="11c3cc6a-d95d-4c2c-9df9-c552aedd85bb" containerID="6b098848cb8db495c5d2bec55c83690bc1bc2676858530efa07033e879ad280d" exitCode=0 Jul 27 20:51:49.263364 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:49.262917 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9lsk" event={"ID":"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb","Type":"ContainerDied","Data":"6b098848cb8db495c5d2bec55c83690bc1bc2676858530efa07033e879ad280d"} Jul 27 20:51:49.264486 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:49.264464 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5psdr" event={"ID":"78365e64-dcf0-450f-ae1e-d8dc161e3657","Type":"ContainerStarted","Data":"b338a7699f27bba5d8e39f41ddaf87d8b8a7eb6eff03ef0d171b91987e93950e"} Jul 27 20:51:49.585744 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:49.585445 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:49.585908 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:49.585747 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls\") pod \"dns-default-j2k9d\" (UID: \"c1f55707-7360-492b-a75f-f2211a512fd3\") " pod="openshift-dns/dns-default-j2k9d" Jul 27 20:51:49.585908 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:49.585605 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 20:51:49.585908 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:49.585823 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7655cf7f49-ldmrw: secret "image-registry-tls" not found Jul 27 20:51:49.585908 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:49.585880 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 20:51:49.586104 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:49.585939 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls podName:c1f55707-7360-492b-a75f-f2211a512fd3 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:51.585920081 +0000 UTC m=+40.109037795 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls") pod "dns-default-j2k9d" (UID: "c1f55707-7360-492b-a75f-f2211a512fd3") : secret "dns-default-metrics-tls" not found Jul 27 20:51:49.586104 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:49.585970 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls podName:43cb7481-cfd4-4480-8523-3b2907b10931 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:51.58594858 +0000 UTC m=+40.109066296 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls") pod "image-registry-7655cf7f49-ldmrw" (UID: "43cb7481-cfd4-4480-8523-3b2907b10931") : secret "image-registry-tls" not found Jul 27 20:51:50.271392 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:50.271352 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h9lsk" event={"ID":"11c3cc6a-d95d-4c2c-9df9-c552aedd85bb","Type":"ContainerStarted","Data":"292752a39aff7fcbef8d987c2525351ee3793b5e6275b92640e784c6ce68d1f4"} Jul 27 20:51:50.300802 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:50.300748 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-h9lsk" podStartSLOduration=8.738090639 podStartE2EDuration="38.300729196s" totalImagesPullingTime="29.562638557s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:51:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:51:50.298583805 +0000 UTC m=+38.821701530" watchObservedRunningTime="2026-07-27 20:51:50.300729196 +0000 UTC m=+38.823846917" Jul 27 20:51:51.275605 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:51.275571 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5psdr" event={"ID":"78365e64-dcf0-450f-ae1e-d8dc161e3657","Type":"ContainerStarted","Data":"cb390d80e08efc67e3a66a707eaf3df99580f25c8835b27c3c7364eb3322e6fb"} Jul 27 20:51:51.295876 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:51.295827 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5psdr" podStartSLOduration=1.8052632929999999 podStartE2EDuration="4.295812749s" totalImagesPullingTime="2.490549456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:51:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:51:51.295320418 +0000 UTC m=+39.818438144" watchObservedRunningTime="2026-07-27 20:51:51.295812749 +0000 UTC m=+39.818930449" Jul 27 20:51:51.599956 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:51.599880 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:51.599956 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:51.599940 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls\") pod \"dns-default-j2k9d\" (UID: \"c1f55707-7360-492b-a75f-f2211a512fd3\") " pod="openshift-dns/dns-default-j2k9d" Jul 27 20:51:51.600125 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:51.600041 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 20:51:51.600125 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:51.600053 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 20:51:51.600125 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:51.600062 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7655cf7f49-ldmrw: secret "image-registry-tls" not found Jul 27 20:51:51.600125 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:51.600114 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls podName:43cb7481-cfd4-4480-8523-3b2907b10931 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:55.600097091 +0000 UTC m=+44.123214815 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls") pod "image-registry-7655cf7f49-ldmrw" (UID: "43cb7481-cfd4-4480-8523-3b2907b10931") : secret "image-registry-tls" not found Jul 27 20:51:51.600125 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:51.600127 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls podName:c1f55707-7360-492b-a75f-f2211a512fd3 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:55.600121208 +0000 UTC m=+44.123238908 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls") pod "dns-default-j2k9d" (UID: "c1f55707-7360-492b-a75f-f2211a512fd3") : secret "dns-default-metrics-tls" not found Jul 27 20:51:54.194877 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.194845 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9"] Jul 27 20:51:54.220510 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.220485 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k67sh"] Jul 27 20:51:54.220644 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.220609 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9" Jul 27 20:51:54.223215 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.223196 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 27 20:51:54.223439 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.223425 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-mlz84\"" Jul 27 20:51:54.223489 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.223467 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 27 20:51:54.224361 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.224346 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 27 20:51:54.237903 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.237877 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k67sh"] Jul 27 20:51:54.237903 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.237906 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9"] Jul 27 20:51:54.238073 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.238008 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k67sh" Jul 27 20:51:54.240917 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.240895 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-4p4qp\"" Jul 27 20:51:54.241028 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.240952 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 27 20:51:54.241028 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.240975 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 27 20:51:54.281790 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.281764 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-9h98q"] Jul 27 20:51:54.305947 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.305911 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-z72h7"] Jul 27 20:51:54.306108 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.306018 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9h98q" Jul 27 20:51:54.309946 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.309920 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 27 20:51:54.310084 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.310006 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9t85j\"" Jul 27 20:51:54.310084 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.310050 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 27 20:51:54.319055 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.319030 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5305b73e-35d4-43dc-9b55-e8e8476d783a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-glfr9\" (UID: \"5305b73e-35d4-43dc-9b55-e8e8476d783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9" Jul 27 20:51:54.319169 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.319076 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxc8t\" (UniqueName: \"kubernetes.io/projected/8633760a-7d53-4b16-bbf0-401f9fa8b8ec-kube-api-access-gxc8t\") pod \"volume-data-source-validator-549c6d54cf-k67sh\" (UID: \"8633760a-7d53-4b16-bbf0-401f9fa8b8ec\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k67sh" Jul 27 20:51:54.319225 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.319196 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhf7r\" (UniqueName: \"kubernetes.io/projected/5305b73e-35d4-43dc-9b55-e8e8476d783a-kube-api-access-xhf7r\") pod \"cluster-samples-operator-6748489656-glfr9\" (UID: \"5305b73e-35d4-43dc-9b55-e8e8476d783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9" Jul 27 20:51:54.330340 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.330309 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-9chx2"] Jul 27 20:51:54.330446 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.330429 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-z72h7" Jul 27 20:51:54.333200 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.333171 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-68jbs\"" Jul 27 20:51:54.354230 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.354196 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-9h98q"] Jul 27 20:51:54.354230 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.354253 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-z72h7"] Jul 27 20:51:54.354436 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.354275 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-9chx2"] Jul 27 20:51:54.354436 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.354312 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-9chx2" Jul 27 20:51:54.357518 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.357497 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 27 20:51:54.357518 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.357515 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 27 20:51:54.357716 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.357546 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-qmd5d\"" Jul 27 20:51:54.357716 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.357515 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 27 20:51:54.357788 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.357739 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 27 20:51:54.419986 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.419947 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9736f967-2f63-40e7-966e-628f22fe91d6-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-9h98q\" (UID: \"9736f967-2f63-40e7-966e-628f22fe91d6\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9h98q" Jul 27 20:51:54.419986 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.419987 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9h98q\" (UID: \"9736f967-2f63-40e7-966e-628f22fe91d6\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9h98q" Jul 27 20:51:54.420228 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.420044 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzlg9\" (UniqueName: \"kubernetes.io/projected/774527c1-f237-40e5-bbb4-31b475608b0b-kube-api-access-qzlg9\") pod \"network-check-source-7d88cb55d8-z72h7\" (UID: \"774527c1-f237-40e5-bbb4-31b475608b0b\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-z72h7" Jul 27 20:51:54.420228 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.420085 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhf7r\" (UniqueName: \"kubernetes.io/projected/5305b73e-35d4-43dc-9b55-e8e8476d783a-kube-api-access-xhf7r\") pod \"cluster-samples-operator-6748489656-glfr9\" (UID: \"5305b73e-35d4-43dc-9b55-e8e8476d783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9" Jul 27 20:51:54.420228 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.420172 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5305b73e-35d4-43dc-9b55-e8e8476d783a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-glfr9\" (UID: \"5305b73e-35d4-43dc-9b55-e8e8476d783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9" Jul 27 20:51:54.420413 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:54.420264 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 20:51:54.420413 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:54.420342 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5305b73e-35d4-43dc-9b55-e8e8476d783a-samples-operator-tls podName:5305b73e-35d4-43dc-9b55-e8e8476d783a nodeName:}" failed. No retries permitted until 2026-07-27 20:51:54.92032512 +0000 UTC m=+43.443442840 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5305b73e-35d4-43dc-9b55-e8e8476d783a-samples-operator-tls") pod "cluster-samples-operator-6748489656-glfr9" (UID: "5305b73e-35d4-43dc-9b55-e8e8476d783a") : secret "samples-operator-tls" not found Jul 27 20:51:54.420413 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.420262 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxc8t\" (UniqueName: \"kubernetes.io/projected/8633760a-7d53-4b16-bbf0-401f9fa8b8ec-kube-api-access-gxc8t\") pod \"volume-data-source-validator-549c6d54cf-k67sh\" (UID: \"8633760a-7d53-4b16-bbf0-401f9fa8b8ec\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k67sh" Jul 27 20:51:54.433036 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.433009 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxc8t\" (UniqueName: \"kubernetes.io/projected/8633760a-7d53-4b16-bbf0-401f9fa8b8ec-kube-api-access-gxc8t\") pod \"volume-data-source-validator-549c6d54cf-k67sh\" (UID: \"8633760a-7d53-4b16-bbf0-401f9fa8b8ec\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k67sh" Jul 27 20:51:54.433170 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.433126 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhf7r\" (UniqueName: \"kubernetes.io/projected/5305b73e-35d4-43dc-9b55-e8e8476d783a-kube-api-access-xhf7r\") pod \"cluster-samples-operator-6748489656-glfr9\" (UID: \"5305b73e-35d4-43dc-9b55-e8e8476d783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9" Jul 27 20:51:54.521290 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.521174 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfrx7\" (UniqueName: \"kubernetes.io/projected/3f79a474-2b0c-4996-876d-ab959ea1f2d9-kube-api-access-dfrx7\") pod \"service-ca-operator-5d6c75f84f-9chx2\" (UID: \"3f79a474-2b0c-4996-876d-ab959ea1f2d9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-9chx2" Jul 27 20:51:54.521433 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.521319 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3f79a474-2b0c-4996-876d-ab959ea1f2d9-config\") pod \"service-ca-operator-5d6c75f84f-9chx2\" (UID: \"3f79a474-2b0c-4996-876d-ab959ea1f2d9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-9chx2" Jul 27 20:51:54.521433 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.521376 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9736f967-2f63-40e7-966e-628f22fe91d6-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-9h98q\" (UID: \"9736f967-2f63-40e7-966e-628f22fe91d6\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9h98q" Jul 27 20:51:54.521433 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.521400 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9h98q\" (UID: \"9736f967-2f63-40e7-966e-628f22fe91d6\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9h98q" Jul 27 20:51:54.521433 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.521427 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzlg9\" (UniqueName: \"kubernetes.io/projected/774527c1-f237-40e5-bbb4-31b475608b0b-kube-api-access-qzlg9\") pod \"network-check-source-7d88cb55d8-z72h7\" (UID: \"774527c1-f237-40e5-bbb4-31b475608b0b\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-z72h7" Jul 27 20:51:54.521556 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.521450 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3f79a474-2b0c-4996-876d-ab959ea1f2d9-serving-cert\") pod \"service-ca-operator-5d6c75f84f-9chx2\" (UID: \"3f79a474-2b0c-4996-876d-ab959ea1f2d9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-9chx2" Jul 27 20:51:54.521556 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:54.521540 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 27 20:51:54.521617 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:54.521602 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert podName:9736f967-2f63-40e7-966e-628f22fe91d6 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:55.02158865 +0000 UTC m=+43.544706350 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9h98q" (UID: "9736f967-2f63-40e7-966e-628f22fe91d6") : secret "networking-console-plugin-cert" not found Jul 27 20:51:54.522035 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.522018 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9736f967-2f63-40e7-966e-628f22fe91d6-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-9h98q\" (UID: \"9736f967-2f63-40e7-966e-628f22fe91d6\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9h98q" Jul 27 20:51:54.533104 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.533069 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzlg9\" (UniqueName: \"kubernetes.io/projected/774527c1-f237-40e5-bbb4-31b475608b0b-kube-api-access-qzlg9\") pod \"network-check-source-7d88cb55d8-z72h7\" (UID: \"774527c1-f237-40e5-bbb4-31b475608b0b\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-z72h7" Jul 27 20:51:54.546728 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.546702 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k67sh" Jul 27 20:51:54.622542 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.622504 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3f79a474-2b0c-4996-876d-ab959ea1f2d9-serving-cert\") pod \"service-ca-operator-5d6c75f84f-9chx2\" (UID: \"3f79a474-2b0c-4996-876d-ab959ea1f2d9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-9chx2" Jul 27 20:51:54.622726 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.622558 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfrx7\" (UniqueName: \"kubernetes.io/projected/3f79a474-2b0c-4996-876d-ab959ea1f2d9-kube-api-access-dfrx7\") pod \"service-ca-operator-5d6c75f84f-9chx2\" (UID: \"3f79a474-2b0c-4996-876d-ab959ea1f2d9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-9chx2" Jul 27 20:51:54.622726 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.622623 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3f79a474-2b0c-4996-876d-ab959ea1f2d9-config\") pod \"service-ca-operator-5d6c75f84f-9chx2\" (UID: \"3f79a474-2b0c-4996-876d-ab959ea1f2d9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-9chx2" Jul 27 20:51:54.623228 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.623202 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3f79a474-2b0c-4996-876d-ab959ea1f2d9-config\") pod \"service-ca-operator-5d6c75f84f-9chx2\" (UID: \"3f79a474-2b0c-4996-876d-ab959ea1f2d9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-9chx2" Jul 27 20:51:54.625337 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.625254 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3f79a474-2b0c-4996-876d-ab959ea1f2d9-serving-cert\") pod \"service-ca-operator-5d6c75f84f-9chx2\" (UID: \"3f79a474-2b0c-4996-876d-ab959ea1f2d9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-9chx2" Jul 27 20:51:54.631060 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.631011 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfrx7\" (UniqueName: \"kubernetes.io/projected/3f79a474-2b0c-4996-876d-ab959ea1f2d9-kube-api-access-dfrx7\") pod \"service-ca-operator-5d6c75f84f-9chx2\" (UID: \"3f79a474-2b0c-4996-876d-ab959ea1f2d9\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-9chx2" Jul 27 20:51:54.639274 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.638990 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-z72h7" Jul 27 20:51:54.662557 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.662496 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-9chx2" Jul 27 20:51:54.703131 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.703104 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k67sh"] Jul 27 20:51:54.705815 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:54.705791 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8633760a_7d53_4b16_bbf0_401f9fa8b8ec.slice/crio-662282ac6d2aedb4a14bc009c6f9d1d7150e05e65675c6b1c56c212785a1fdad WatchSource:0}: Error finding container 662282ac6d2aedb4a14bc009c6f9d1d7150e05e65675c6b1c56c212785a1fdad: Status 404 returned error can't find the container with id 662282ac6d2aedb4a14bc009c6f9d1d7150e05e65675c6b1c56c212785a1fdad Jul 27 20:51:54.804715 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.804684 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-z72h7"] Jul 27 20:51:54.807964 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:54.807917 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod774527c1_f237_40e5_bbb4_31b475608b0b.slice/crio-1d2648875767f902c7a99c0032e9b9063172f253c0c4d8d1d029257436a063c9 WatchSource:0}: Error finding container 1d2648875767f902c7a99c0032e9b9063172f253c0c4d8d1d029257436a063c9: Status 404 returned error can't find the container with id 1d2648875767f902c7a99c0032e9b9063172f253c0c4d8d1d029257436a063c9 Jul 27 20:51:54.848874 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.848842 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-9chx2"] Jul 27 20:51:54.852586 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:51:54.852558 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f79a474_2b0c_4996_876d_ab959ea1f2d9.slice/crio-8e866d182388940db9d0c4f4cf2ec0dc591e6a25a3624a92a007e445136a8b57 WatchSource:0}: Error finding container 8e866d182388940db9d0c4f4cf2ec0dc591e6a25a3624a92a007e445136a8b57: Status 404 returned error can't find the container with id 8e866d182388940db9d0c4f4cf2ec0dc591e6a25a3624a92a007e445136a8b57 Jul 27 20:51:54.924853 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:54.924817 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5305b73e-35d4-43dc-9b55-e8e8476d783a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-glfr9\" (UID: \"5305b73e-35d4-43dc-9b55-e8e8476d783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9" Jul 27 20:51:54.924985 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:54.924965 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 20:51:54.925054 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:54.925044 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5305b73e-35d4-43dc-9b55-e8e8476d783a-samples-operator-tls podName:5305b73e-35d4-43dc-9b55-e8e8476d783a nodeName:}" failed. No retries permitted until 2026-07-27 20:51:55.925028052 +0000 UTC m=+44.448145753 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5305b73e-35d4-43dc-9b55-e8e8476d783a-samples-operator-tls") pod "cluster-samples-operator-6748489656-glfr9" (UID: "5305b73e-35d4-43dc-9b55-e8e8476d783a") : secret "samples-operator-tls" not found Jul 27 20:51:55.025410 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:55.025370 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9h98q\" (UID: \"9736f967-2f63-40e7-966e-628f22fe91d6\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9h98q" Jul 27 20:51:55.025543 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:55.025515 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 27 20:51:55.025601 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:55.025578 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert podName:9736f967-2f63-40e7-966e-628f22fe91d6 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:56.025562416 +0000 UTC m=+44.548680116 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9h98q" (UID: "9736f967-2f63-40e7-966e-628f22fe91d6") : secret "networking-console-plugin-cert" not found Jul 27 20:51:55.285603 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:55.285564 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-9chx2" event={"ID":"3f79a474-2b0c-4996-876d-ab959ea1f2d9","Type":"ContainerStarted","Data":"8e866d182388940db9d0c4f4cf2ec0dc591e6a25a3624a92a007e445136a8b57"} Jul 27 20:51:55.286613 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:55.286589 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-z72h7" event={"ID":"774527c1-f237-40e5-bbb4-31b475608b0b","Type":"ContainerStarted","Data":"1d2648875767f902c7a99c0032e9b9063172f253c0c4d8d1d029257436a063c9"} Jul 27 20:51:55.287567 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:55.287542 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k67sh" event={"ID":"8633760a-7d53-4b16-bbf0-401f9fa8b8ec","Type":"ContainerStarted","Data":"662282ac6d2aedb4a14bc009c6f9d1d7150e05e65675c6b1c56c212785a1fdad"} Jul 27 20:51:55.630429 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:55.630335 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:51:55.630623 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:55.630426 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls\") pod \"dns-default-j2k9d\" (UID: \"c1f55707-7360-492b-a75f-f2211a512fd3\") " pod="openshift-dns/dns-default-j2k9d" Jul 27 20:51:55.630623 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:55.630531 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 20:51:55.630623 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:55.630555 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7655cf7f49-ldmrw: secret "image-registry-tls" not found Jul 27 20:51:55.630623 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:55.630563 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 20:51:55.630623 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:55.630620 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls podName:c1f55707-7360-492b-a75f-f2211a512fd3 nodeName:}" failed. No retries permitted until 2026-07-27 20:52:03.630600992 +0000 UTC m=+52.153718694 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls") pod "dns-default-j2k9d" (UID: "c1f55707-7360-492b-a75f-f2211a512fd3") : secret "dns-default-metrics-tls" not found Jul 27 20:51:55.630885 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:55.630638 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls podName:43cb7481-cfd4-4480-8523-3b2907b10931 nodeName:}" failed. No retries permitted until 2026-07-27 20:52:03.630628977 +0000 UTC m=+52.153746682 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls") pod "image-registry-7655cf7f49-ldmrw" (UID: "43cb7481-cfd4-4480-8523-3b2907b10931") : secret "image-registry-tls" not found Jul 27 20:51:55.932246 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:55.932197 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5305b73e-35d4-43dc-9b55-e8e8476d783a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-glfr9\" (UID: \"5305b73e-35d4-43dc-9b55-e8e8476d783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9" Jul 27 20:51:55.932417 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:55.932366 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 20:51:55.932473 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:55.932423 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5305b73e-35d4-43dc-9b55-e8e8476d783a-samples-operator-tls podName:5305b73e-35d4-43dc-9b55-e8e8476d783a nodeName:}" failed. No retries permitted until 2026-07-27 20:51:57.932409617 +0000 UTC m=+46.455527321 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5305b73e-35d4-43dc-9b55-e8e8476d783a-samples-operator-tls") pod "cluster-samples-operator-6748489656-glfr9" (UID: "5305b73e-35d4-43dc-9b55-e8e8476d783a") : secret "samples-operator-tls" not found Jul 27 20:51:56.033377 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:56.033319 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9h98q\" (UID: \"9736f967-2f63-40e7-966e-628f22fe91d6\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9h98q" Jul 27 20:51:56.033573 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:56.033521 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 27 20:51:56.033647 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:56.033628 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert podName:9736f967-2f63-40e7-966e-628f22fe91d6 nodeName:}" failed. No retries permitted until 2026-07-27 20:51:58.033604464 +0000 UTC m=+46.556722186 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9h98q" (UID: "9736f967-2f63-40e7-966e-628f22fe91d6") : secret "networking-console-plugin-cert" not found Jul 27 20:51:57.950562 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:57.950521 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5305b73e-35d4-43dc-9b55-e8e8476d783a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-glfr9\" (UID: \"5305b73e-35d4-43dc-9b55-e8e8476d783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9" Jul 27 20:51:57.950994 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:57.950694 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 20:51:57.950994 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:57.950779 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5305b73e-35d4-43dc-9b55-e8e8476d783a-samples-operator-tls podName:5305b73e-35d4-43dc-9b55-e8e8476d783a nodeName:}" failed. No retries permitted until 2026-07-27 20:52:01.950756828 +0000 UTC m=+50.473874546 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5305b73e-35d4-43dc-9b55-e8e8476d783a-samples-operator-tls") pod "cluster-samples-operator-6748489656-glfr9" (UID: "5305b73e-35d4-43dc-9b55-e8e8476d783a") : secret "samples-operator-tls" not found Jul 27 20:51:58.051759 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:58.051722 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9h98q\" (UID: \"9736f967-2f63-40e7-966e-628f22fe91d6\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9h98q" Jul 27 20:51:58.051918 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:58.051854 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 27 20:51:58.051918 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:51:58.051910 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert podName:9736f967-2f63-40e7-966e-628f22fe91d6 nodeName:}" failed. No retries permitted until 2026-07-27 20:52:02.051890767 +0000 UTC m=+50.575008471 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9h98q" (UID: "9736f967-2f63-40e7-966e-628f22fe91d6") : secret "networking-console-plugin-cert" not found Jul 27 20:51:59.298748 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:59.298707 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k67sh" event={"ID":"8633760a-7d53-4b16-bbf0-401f9fa8b8ec","Type":"ContainerStarted","Data":"2a2d7d8dcbefbb14ad58e9ac59221349e473acdbc90a928163a4ebdffd3ca111"} Jul 27 20:51:59.300149 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:59.300118 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-9chx2" event={"ID":"3f79a474-2b0c-4996-876d-ab959ea1f2d9","Type":"ContainerStarted","Data":"82a37d920b4c8f7452df0ecc64684260c8d54a7494d5ca6db07b15932159026d"} Jul 27 20:51:59.301658 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:59.301636 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-z72h7" event={"ID":"774527c1-f237-40e5-bbb4-31b475608b0b","Type":"ContainerStarted","Data":"70ff5f08182c2b9a9d9ae8614b6774d311aef1347a5cd128a7d13203969fbb6f"} Jul 27 20:51:59.317452 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:59.317404 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k67sh" podStartSLOduration=1.428618657 podStartE2EDuration="5.317389341s" totalImagesPullingTime="3.888770684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:51:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:51:59.315965595 +0000 UTC m=+47.839083317" watchObservedRunningTime="2026-07-27 20:51:59.317389341 +0000 UTC m=+47.840507082" Jul 27 20:51:59.336063 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:59.336014 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-z72h7" podStartSLOduration=1.495444817 podStartE2EDuration="5.335998355s" totalImagesPullingTime="3.840553538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:51:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:51:59.33481945 +0000 UTC m=+47.857937173" watchObservedRunningTime="2026-07-27 20:51:59.335998355 +0000 UTC m=+47.859116080" Jul 27 20:51:59.358839 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:51:59.358787 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-9chx2" podStartSLOduration=1.572173451 podStartE2EDuration="5.358769109s" totalImagesPullingTime="3.786595658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:51:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:51:59.35832936 +0000 UTC m=+47.881447083" watchObservedRunningTime="2026-07-27 20:51:59.358769109 +0000 UTC m=+47.881886830" Jul 27 20:52:00.114198 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.114158 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4bhvm"] Jul 27 20:52:00.125171 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.125145 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:00.129953 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.129924 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qwd8q\"" Jul 27 20:52:00.130109 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.129924 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 27 20:52:00.130109 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.129927 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 27 20:52:00.130734 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.130715 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 27 20:52:00.130851 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.130715 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 27 20:52:00.132009 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.131990 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4bhvm"] Jul 27 20:52:00.269356 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.269322 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7db41ad-fec7-4420-93d8-869d4ee1965b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:00.269530 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.269397 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7db41ad-fec7-4420-93d8-869d4ee1965b-data-volume\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:00.269530 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.269417 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44j4j\" (UniqueName: \"kubernetes.io/projected/f7db41ad-fec7-4420-93d8-869d4ee1965b-kube-api-access-44j4j\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:00.269530 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.269435 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f7db41ad-fec7-4420-93d8-869d4ee1965b-crio-socket\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:00.269530 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.269452 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7db41ad-fec7-4420-93d8-869d4ee1965b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:00.370637 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.370553 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7db41ad-fec7-4420-93d8-869d4ee1965b-data-volume\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:00.370637 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.370587 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44j4j\" (UniqueName: \"kubernetes.io/projected/f7db41ad-fec7-4420-93d8-869d4ee1965b-kube-api-access-44j4j\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:00.370637 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.370608 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f7db41ad-fec7-4420-93d8-869d4ee1965b-crio-socket\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:00.370637 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.370626 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7db41ad-fec7-4420-93d8-869d4ee1965b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:00.371184 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.370682 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7db41ad-fec7-4420-93d8-869d4ee1965b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:00.371184 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:00.370755 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 20:52:00.371184 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:00.370813 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7db41ad-fec7-4420-93d8-869d4ee1965b-kube-rbac-proxy-cm podName:f7db41ad-fec7-4420-93d8-869d4ee1965b nodeName:}" failed. No retries permitted until 2026-07-27 20:52:00.87079854 +0000 UTC m=+49.393916252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f7db41ad-fec7-4420-93d8-869d4ee1965b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4bhvm" (UID: "f7db41ad-fec7-4420-93d8-869d4ee1965b") : configmap "kube-rbac-proxy" not found Jul 27 20:52:00.371184 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:00.370827 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 20:52:00.371184 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.370854 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f7db41ad-fec7-4420-93d8-869d4ee1965b-crio-socket\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:00.371184 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:00.370909 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7db41ad-fec7-4420-93d8-869d4ee1965b-insights-runtime-extractor-tls podName:f7db41ad-fec7-4420-93d8-869d4ee1965b nodeName:}" failed. No retries permitted until 2026-07-27 20:52:00.870889773 +0000 UTC m=+49.394007476 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f7db41ad-fec7-4420-93d8-869d4ee1965b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4bhvm" (UID: "f7db41ad-fec7-4420-93d8-869d4ee1965b") : secret "insights-runtime-extractor-tls" not found Jul 27 20:52:00.371184 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.370955 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7db41ad-fec7-4420-93d8-869d4ee1965b-data-volume\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:00.398255 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.398213 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44j4j\" (UniqueName: \"kubernetes.io/projected/f7db41ad-fec7-4420-93d8-869d4ee1965b-kube-api-access-44j4j\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:00.874881 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.874840 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7db41ad-fec7-4420-93d8-869d4ee1965b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:00.875036 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:00.874919 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7db41ad-fec7-4420-93d8-869d4ee1965b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:00.875036 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:00.875001 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 20:52:00.875036 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:00.875003 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 20:52:00.875154 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:00.875070 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7db41ad-fec7-4420-93d8-869d4ee1965b-kube-rbac-proxy-cm podName:f7db41ad-fec7-4420-93d8-869d4ee1965b nodeName:}" failed. No retries permitted until 2026-07-27 20:52:01.875052339 +0000 UTC m=+50.398170042 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f7db41ad-fec7-4420-93d8-869d4ee1965b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4bhvm" (UID: "f7db41ad-fec7-4420-93d8-869d4ee1965b") : configmap "kube-rbac-proxy" not found Jul 27 20:52:00.875154 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:00.875086 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7db41ad-fec7-4420-93d8-869d4ee1965b-insights-runtime-extractor-tls podName:f7db41ad-fec7-4420-93d8-869d4ee1965b nodeName:}" failed. No retries permitted until 2026-07-27 20:52:01.875078323 +0000 UTC m=+50.398196023 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f7db41ad-fec7-4420-93d8-869d4ee1965b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4bhvm" (UID: "f7db41ad-fec7-4420-93d8-869d4ee1965b") : secret "insights-runtime-extractor-tls" not found Jul 27 20:52:01.882922 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:01.882880 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7db41ad-fec7-4420-93d8-869d4ee1965b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:01.883303 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:01.882993 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7db41ad-fec7-4420-93d8-869d4ee1965b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:01.883303 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:01.883038 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 20:52:01.883303 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:01.883097 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 20:52:01.883303 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:01.883105 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7db41ad-fec7-4420-93d8-869d4ee1965b-insights-runtime-extractor-tls podName:f7db41ad-fec7-4420-93d8-869d4ee1965b nodeName:}" failed. No retries permitted until 2026-07-27 20:52:03.883089902 +0000 UTC m=+52.406207602 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f7db41ad-fec7-4420-93d8-869d4ee1965b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4bhvm" (UID: "f7db41ad-fec7-4420-93d8-869d4ee1965b") : secret "insights-runtime-extractor-tls" not found Jul 27 20:52:01.883303 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:01.883153 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7db41ad-fec7-4420-93d8-869d4ee1965b-kube-rbac-proxy-cm podName:f7db41ad-fec7-4420-93d8-869d4ee1965b nodeName:}" failed. No retries permitted until 2026-07-27 20:52:03.883141073 +0000 UTC m=+52.406258773 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f7db41ad-fec7-4420-93d8-869d4ee1965b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4bhvm" (UID: "f7db41ad-fec7-4420-93d8-869d4ee1965b") : configmap "kube-rbac-proxy" not found Jul 27 20:52:01.984477 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:01.984437 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5305b73e-35d4-43dc-9b55-e8e8476d783a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-glfr9\" (UID: \"5305b73e-35d4-43dc-9b55-e8e8476d783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9" Jul 27 20:52:01.984628 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:01.984580 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 20:52:01.984664 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:01.984647 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5305b73e-35d4-43dc-9b55-e8e8476d783a-samples-operator-tls podName:5305b73e-35d4-43dc-9b55-e8e8476d783a nodeName:}" failed. No retries permitted until 2026-07-27 20:52:09.984632213 +0000 UTC m=+58.507749913 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5305b73e-35d4-43dc-9b55-e8e8476d783a-samples-operator-tls") pod "cluster-samples-operator-6748489656-glfr9" (UID: "5305b73e-35d4-43dc-9b55-e8e8476d783a") : secret "samples-operator-tls" not found Jul 27 20:52:02.085651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:02.085616 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9h98q\" (UID: \"9736f967-2f63-40e7-966e-628f22fe91d6\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9h98q" Jul 27 20:52:02.085818 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:02.085803 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 27 20:52:02.085906 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:02.085879 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert podName:9736f967-2f63-40e7-966e-628f22fe91d6 nodeName:}" failed. No retries permitted until 2026-07-27 20:52:10.085859525 +0000 UTC m=+58.608977226 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9h98q" (UID: "9736f967-2f63-40e7-966e-628f22fe91d6") : secret "networking-console-plugin-cert" not found Jul 27 20:52:02.103075 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:02.103049 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lbt9p_78a12024-7c5c-4f1c-a15d-58876f88ee45/dns-node-resolver/0.log" Jul 27 20:52:03.102854 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:03.102824 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-74fh5_3705087b-83f8-4e75-84e2-ce01936e4657/node-ca/0.log" Jul 27 20:52:03.701118 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:03.701062 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:52:03.701389 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:03.701129 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls\") pod \"dns-default-j2k9d\" (UID: \"c1f55707-7360-492b-a75f-f2211a512fd3\") " pod="openshift-dns/dns-default-j2k9d" Jul 27 20:52:03.701389 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:03.701221 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 20:52:03.701389 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:03.701257 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7655cf7f49-ldmrw: secret "image-registry-tls" not found Jul 27 20:52:03.701389 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:03.701287 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 20:52:03.701389 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:03.701326 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls podName:43cb7481-cfd4-4480-8523-3b2907b10931 nodeName:}" failed. No retries permitted until 2026-07-27 20:52:19.701310836 +0000 UTC m=+68.224428537 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls") pod "image-registry-7655cf7f49-ldmrw" (UID: "43cb7481-cfd4-4480-8523-3b2907b10931") : secret "image-registry-tls" not found Jul 27 20:52:03.701389 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:03.701344 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls podName:c1f55707-7360-492b-a75f-f2211a512fd3 nodeName:}" failed. No retries permitted until 2026-07-27 20:52:19.701331666 +0000 UTC m=+68.224449365 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls") pod "dns-default-j2k9d" (UID: "c1f55707-7360-492b-a75f-f2211a512fd3") : secret "dns-default-metrics-tls" not found Jul 27 20:52:03.902518 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:03.902482 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7db41ad-fec7-4420-93d8-869d4ee1965b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:03.902817 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:03.902565 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7db41ad-fec7-4420-93d8-869d4ee1965b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:03.902817 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:03.902658 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 20:52:03.902817 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:03.902738 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7db41ad-fec7-4420-93d8-869d4ee1965b-insights-runtime-extractor-tls podName:f7db41ad-fec7-4420-93d8-869d4ee1965b nodeName:}" failed. No retries permitted until 2026-07-27 20:52:07.902718103 +0000 UTC m=+56.425835805 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f7db41ad-fec7-4420-93d8-869d4ee1965b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4bhvm" (UID: "f7db41ad-fec7-4420-93d8-869d4ee1965b") : secret "insights-runtime-extractor-tls" not found Jul 27 20:52:03.903105 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:03.903085 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7db41ad-fec7-4420-93d8-869d4ee1965b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:03.904257 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:03.904219 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5psdr_78365e64-dcf0-450f-ae1e-d8dc161e3657/serve-healthcheck-canary/0.log" Jul 27 20:52:07.930832 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:07.930774 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7db41ad-fec7-4420-93d8-869d4ee1965b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:07.931220 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:07.930938 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 20:52:07.931220 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:07.931001 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7db41ad-fec7-4420-93d8-869d4ee1965b-insights-runtime-extractor-tls podName:f7db41ad-fec7-4420-93d8-869d4ee1965b nodeName:}" failed. No retries permitted until 2026-07-27 20:52:15.930986252 +0000 UTC m=+64.454103951 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f7db41ad-fec7-4420-93d8-869d4ee1965b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4bhvm" (UID: "f7db41ad-fec7-4420-93d8-869d4ee1965b") : secret "insights-runtime-extractor-tls" not found Jul 27 20:52:10.043568 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:10.043529 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5305b73e-35d4-43dc-9b55-e8e8476d783a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-glfr9\" (UID: \"5305b73e-35d4-43dc-9b55-e8e8476d783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9" Jul 27 20:52:10.046052 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:10.046030 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5305b73e-35d4-43dc-9b55-e8e8476d783a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-glfr9\" (UID: \"5305b73e-35d4-43dc-9b55-e8e8476d783a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9" Jul 27 20:52:10.129187 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:10.129161 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9" Jul 27 20:52:10.144567 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:10.144537 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9h98q\" (UID: \"9736f967-2f63-40e7-966e-628f22fe91d6\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9h98q" Jul 27 20:52:10.144706 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:10.144689 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 27 20:52:10.144763 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:10.144754 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert podName:9736f967-2f63-40e7-966e-628f22fe91d6 nodeName:}" failed. No retries permitted until 2026-07-27 20:52:26.144738281 +0000 UTC m=+74.667855982 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9h98q" (UID: "9736f967-2f63-40e7-966e-628f22fe91d6") : secret "networking-console-plugin-cert" not found Jul 27 20:52:10.290027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:10.289984 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9"] Jul 27 20:52:11.332524 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:11.332491 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9" event={"ID":"5305b73e-35d4-43dc-9b55-e8e8476d783a","Type":"ContainerStarted","Data":"508e7e65f7fc9f2c9dbdae910950d69a20adafe3916704bee2d970ea9c21adb2"} Jul 27 20:52:12.337686 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:12.337648 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9" event={"ID":"5305b73e-35d4-43dc-9b55-e8e8476d783a","Type":"ContainerStarted","Data":"d76acdbc47ec86a0648904711e54a89ab0fab8557ad5c0c7ae955980b1b89f8c"} Jul 27 20:52:13.343162 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:13.343124 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9" event={"ID":"5305b73e-35d4-43dc-9b55-e8e8476d783a","Type":"ContainerStarted","Data":"be7d4967a8a702af8f3735c3bc8e88c03b7ad8268d8bb7dccbd8e4c14af3a88f"} Jul 27 20:52:13.360211 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:13.360135 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-glfr9" podStartSLOduration=17.513424132 podStartE2EDuration="19.360119701s" totalImagesPullingTime="1.846695569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:51:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:52:13.359132571 +0000 UTC m=+61.882250292" watchObservedRunningTime="2026-07-27 20:52:13.360119701 +0000 UTC m=+61.883237458" Jul 27 20:52:14.252836 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:14.252810 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9gbjx" Jul 27 20:52:15.991479 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:15.991426 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7db41ad-fec7-4420-93d8-869d4ee1965b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:15.993949 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:15.993923 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7db41ad-fec7-4420-93d8-869d4ee1965b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4bhvm\" (UID: \"f7db41ad-fec7-4420-93d8-869d4ee1965b\") " pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:16.038744 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:16.038708 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qwd8q\"" Jul 27 20:52:16.045481 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:16.045455 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4bhvm" Jul 27 20:52:16.172579 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:16.172559 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4bhvm"] Jul 27 20:52:16.175006 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:52:16.174980 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7db41ad_fec7_4420_93d8_869d4ee1965b.slice/crio-a99d41fd91d274520003e30205cf3a79bb72da2695f80ef93fd79c2cacafc6d8 WatchSource:0}: Error finding container a99d41fd91d274520003e30205cf3a79bb72da2695f80ef93fd79c2cacafc6d8: Status 404 returned error can't find the container with id a99d41fd91d274520003e30205cf3a79bb72da2695f80ef93fd79c2cacafc6d8 Jul 27 20:52:16.355378 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:16.355292 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4bhvm" event={"ID":"f7db41ad-fec7-4420-93d8-869d4ee1965b","Type":"ContainerStarted","Data":"a99d41fd91d274520003e30205cf3a79bb72da2695f80ef93fd79c2cacafc6d8"} Jul 27 20:52:17.706290 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:17.706254 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs\") pod \"network-metrics-daemon-wk44n\" (UID: \"668f5b4e-867c-47c2-b868-cb3f79c14f2a\") " pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:52:17.706722 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:17.706314 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret\") pod \"global-pull-secret-syncer-qzsr7\" (UID: \"86fb8fed-0e82-44c5-9c08-08a9b4d8eea3\") " pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:52:17.709629 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:17.709604 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 27 20:52:17.710874 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:17.710856 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 27 20:52:17.719788 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:17.719759 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/86fb8fed-0e82-44c5-9c08-08a9b4d8eea3-original-pull-secret\") pod \"global-pull-secret-syncer-qzsr7\" (UID: \"86fb8fed-0e82-44c5-9c08-08a9b4d8eea3\") " pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:52:17.719899 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:17.719825 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/668f5b4e-867c-47c2-b868-cb3f79c14f2a-metrics-certs\") pod \"network-metrics-daemon-wk44n\" (UID: \"668f5b4e-867c-47c2-b868-cb3f79c14f2a\") " pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:52:17.807367 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:17.807331 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znkxb\" (UniqueName: \"kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb\") pod \"network-check-target-zt78l\" (UID: \"8add67af-ae5d-4d84-b040-7d8cc145e665\") " pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:52:17.810360 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:17.810336 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znkxb\" (UniqueName: \"kubernetes.io/projected/8add67af-ae5d-4d84-b040-7d8cc145e665-kube-api-access-znkxb\") pod \"network-check-target-zt78l\" (UID: \"8add67af-ae5d-4d84-b040-7d8cc145e665\") " pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:52:17.858104 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:17.858068 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qzsr7" Jul 27 20:52:17.868203 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:17.868171 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2w5wq\"" Jul 27 20:52:17.874921 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:17.874031 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6wczd\"" Jul 27 20:52:17.876045 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:17.876013 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wk44n" Jul 27 20:52:17.882650 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:17.882631 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:52:18.049307 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:52:18.049217 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86fb8fed_0e82_44c5_9c08_08a9b4d8eea3.slice/crio-7c8587c13ee7152be54dbdc5e0ba15dc234c1f8513eede2c7e58122c3269c40b WatchSource:0}: Error finding container 7c8587c13ee7152be54dbdc5e0ba15dc234c1f8513eede2c7e58122c3269c40b: Status 404 returned error can't find the container with id 7c8587c13ee7152be54dbdc5e0ba15dc234c1f8513eede2c7e58122c3269c40b Jul 27 20:52:18.052931 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:18.052913 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qzsr7"] Jul 27 20:52:18.070358 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:18.070326 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wk44n"] Jul 27 20:52:18.074396 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:52:18.074368 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod668f5b4e_867c_47c2_b868_cb3f79c14f2a.slice/crio-8d542945083a3a839a465c98a5f68d3d714ec09a30708bfaa7c9b8aed5a47e04 WatchSource:0}: Error finding container 8d542945083a3a839a465c98a5f68d3d714ec09a30708bfaa7c9b8aed5a47e04: Status 404 returned error can't find the container with id 8d542945083a3a839a465c98a5f68d3d714ec09a30708bfaa7c9b8aed5a47e04 Jul 27 20:52:18.094469 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:18.094445 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zt78l"] Jul 27 20:52:18.097075 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:52:18.097051 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8add67af_ae5d_4d84_b040_7d8cc145e665.slice/crio-ead24bb752f813e436650fc23eb295930e27cc86de953cdb4187ae8d7ff74072 WatchSource:0}: Error finding container ead24bb752f813e436650fc23eb295930e27cc86de953cdb4187ae8d7ff74072: Status 404 returned error can't find the container with id ead24bb752f813e436650fc23eb295930e27cc86de953cdb4187ae8d7ff74072 Jul 27 20:52:18.363631 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:18.363586 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qzsr7" event={"ID":"86fb8fed-0e82-44c5-9c08-08a9b4d8eea3","Type":"ContainerStarted","Data":"7c8587c13ee7152be54dbdc5e0ba15dc234c1f8513eede2c7e58122c3269c40b"} Jul 27 20:52:18.365288 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:18.365168 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zt78l" event={"ID":"8add67af-ae5d-4d84-b040-7d8cc145e665","Type":"ContainerStarted","Data":"f13aba8336c1d2109030bae32c265a9733c3fab440a5ff45a4385ac65080a371"} Jul 27 20:52:18.365288 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:18.365218 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zt78l" event={"ID":"8add67af-ae5d-4d84-b040-7d8cc145e665","Type":"ContainerStarted","Data":"ead24bb752f813e436650fc23eb295930e27cc86de953cdb4187ae8d7ff74072"} Jul 27 20:52:18.365546 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:18.365518 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:52:18.366350 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:18.366325 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wk44n" event={"ID":"668f5b4e-867c-47c2-b868-cb3f79c14f2a","Type":"ContainerStarted","Data":"8d542945083a3a839a465c98a5f68d3d714ec09a30708bfaa7c9b8aed5a47e04"} Jul 27 20:52:18.383145 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:18.383090 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zt78l" podStartSLOduration=66.383067808 podStartE2EDuration="1m6.383067808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:51:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:52:18.38190211 +0000 UTC m=+66.905019833" watchObservedRunningTime="2026-07-27 20:52:18.383067808 +0000 UTC m=+66.906185529" Jul 27 20:52:19.729078 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:19.729043 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls\") pod \"dns-default-j2k9d\" (UID: \"c1f55707-7360-492b-a75f-f2211a512fd3\") " pod="openshift-dns/dns-default-j2k9d" Jul 27 20:52:19.729524 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:19.729163 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:52:19.732390 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:19.732366 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls\") pod \"image-registry-7655cf7f49-ldmrw\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:52:19.732605 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:19.732584 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1f55707-7360-492b-a75f-f2211a512fd3-metrics-tls\") pod \"dns-default-j2k9d\" (UID: \"c1f55707-7360-492b-a75f-f2211a512fd3\") " pod="openshift-dns/dns-default-j2k9d" Jul 27 20:52:19.952817 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:19.952783 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xqs7p\"" Jul 27 20:52:19.960598 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:19.960567 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:52:19.961371 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:19.961351 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2qj4g\"" Jul 27 20:52:19.968803 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:19.968778 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j2k9d" Jul 27 20:52:23.120183 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:23.120156 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7655cf7f49-ldmrw"] Jul 27 20:52:23.130321 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:23.130294 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-j2k9d"] Jul 27 20:52:23.224858 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:52:23.224772 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43cb7481_cfd4_4480_8523_3b2907b10931.slice/crio-cf0e0f69202b4c3be78d5c8d770d00544a1f6ce27e92fc749a3d8f3b2e972f78 WatchSource:0}: Error finding container cf0e0f69202b4c3be78d5c8d770d00544a1f6ce27e92fc749a3d8f3b2e972f78: Status 404 returned error can't find the container with id cf0e0f69202b4c3be78d5c8d770d00544a1f6ce27e92fc749a3d8f3b2e972f78 Jul 27 20:52:23.225062 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:52:23.225035 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1f55707_7360_492b_a75f_f2211a512fd3.slice/crio-912063b783f33faa48ddf11427a7e9bc72696d73fdf7327c0f8ce9ac5a06c24d WatchSource:0}: Error finding container 912063b783f33faa48ddf11427a7e9bc72696d73fdf7327c0f8ce9ac5a06c24d: Status 404 returned error can't find the container with id 912063b783f33faa48ddf11427a7e9bc72696d73fdf7327c0f8ce9ac5a06c24d Jul 27 20:52:23.389533 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:23.389446 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wk44n" event={"ID":"668f5b4e-867c-47c2-b868-cb3f79c14f2a","Type":"ContainerStarted","Data":"cd33e4623b8dae78f7d32dcd57bf504a45062572b4cf4c3a02fa9fa7b68e69fb"} Jul 27 20:52:23.391308 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:23.391279 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4bhvm" event={"ID":"f7db41ad-fec7-4420-93d8-869d4ee1965b","Type":"ContainerStarted","Data":"aa0506d8f05e333c9f19b93928daeff2ccd955c47ff8260f251c0531de79e1a6"} Jul 27 20:52:23.392883 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:23.392852 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qzsr7" event={"ID":"86fb8fed-0e82-44c5-9c08-08a9b4d8eea3","Type":"ContainerStarted","Data":"e23a6ea9dd44f891f477b27878cee7cf3b876b1752615ae1a8e90d8a5e056a61"} Jul 27 20:52:23.395439 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:23.395321 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" event={"ID":"43cb7481-cfd4-4480-8523-3b2907b10931","Type":"ContainerStarted","Data":"4c88354f8a7e80bbd119f6245239d3e947b5a38863835f732ac4d4cfc6125080"} Jul 27 20:52:23.395439 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:23.395370 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:52:23.395439 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:23.395390 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" event={"ID":"43cb7481-cfd4-4480-8523-3b2907b10931","Type":"ContainerStarted","Data":"cf0e0f69202b4c3be78d5c8d770d00544a1f6ce27e92fc749a3d8f3b2e972f78"} Jul 27 20:52:23.398351 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:23.398324 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j2k9d" event={"ID":"c1f55707-7360-492b-a75f-f2211a512fd3","Type":"ContainerStarted","Data":"912063b783f33faa48ddf11427a7e9bc72696d73fdf7327c0f8ce9ac5a06c24d"} Jul 27 20:52:23.412585 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:23.412538 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qzsr7" podStartSLOduration=66.239326218 podStartE2EDuration="1m11.41252297s" totalImagesPullingTime="5.173196752s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:51:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:52:23.412410619 +0000 UTC m=+71.935528376" watchObservedRunningTime="2026-07-27 20:52:23.41252297 +0000 UTC m=+71.935640692" Jul 27 20:52:23.439611 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:23.438860 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" podStartSLOduration=49.438840558 podStartE2EDuration="49.438840558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:51:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:52:23.438230195 +0000 UTC m=+71.961347918" watchObservedRunningTime="2026-07-27 20:52:23.438840558 +0000 UTC m=+71.961958283" Jul 27 20:52:24.002941 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.002904 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc"] Jul 27 20:52:24.006674 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.006637 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.012279 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.012254 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 27 20:52:24.012593 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.012574 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 27 20:52:24.012867 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.012843 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 27 20:52:24.013506 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.013488 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 27 20:52:24.013691 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.013675 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 27 20:52:24.013878 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.013859 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 27 20:52:24.014100 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.014086 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 27 20:52:24.026694 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.026672 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc"] Jul 27 20:52:24.068271 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.068226 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5c5620b6-cae2-4cd1-a1c7-b798cb5951b8-ca\") pod \"cluster-proxy-proxy-agent-777496d5cd-grrtc\" (UID: \"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.068271 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.068277 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c5620b6-cae2-4cd1-a1c7-b798cb5951b8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-777496d5cd-grrtc\" (UID: \"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.068516 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.068311 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5c5620b6-cae2-4cd1-a1c7-b798cb5951b8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-777496d5cd-grrtc\" (UID: \"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.068516 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.068344 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx467\" (UniqueName: \"kubernetes.io/projected/5c5620b6-cae2-4cd1-a1c7-b798cb5951b8-kube-api-access-hx467\") pod \"cluster-proxy-proxy-agent-777496d5cd-grrtc\" (UID: \"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.068516 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.068429 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5c5620b6-cae2-4cd1-a1c7-b798cb5951b8-hub\") pod \"cluster-proxy-proxy-agent-777496d5cd-grrtc\" (UID: \"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.068516 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.068457 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5c5620b6-cae2-4cd1-a1c7-b798cb5951b8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-777496d5cd-grrtc\" (UID: \"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.169569 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.169533 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx467\" (UniqueName: \"kubernetes.io/projected/5c5620b6-cae2-4cd1-a1c7-b798cb5951b8-kube-api-access-hx467\") pod \"cluster-proxy-proxy-agent-777496d5cd-grrtc\" (UID: \"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.169569 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.169610 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5c5620b6-cae2-4cd1-a1c7-b798cb5951b8-hub\") pod \"cluster-proxy-proxy-agent-777496d5cd-grrtc\" (UID: \"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.169569 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.169645 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5c5620b6-cae2-4cd1-a1c7-b798cb5951b8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-777496d5cd-grrtc\" (UID: \"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.170378 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.169695 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5c5620b6-cae2-4cd1-a1c7-b798cb5951b8-ca\") pod \"cluster-proxy-proxy-agent-777496d5cd-grrtc\" (UID: \"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.170378 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.169719 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c5620b6-cae2-4cd1-a1c7-b798cb5951b8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-777496d5cd-grrtc\" (UID: \"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.170378 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.169753 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5c5620b6-cae2-4cd1-a1c7-b798cb5951b8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-777496d5cd-grrtc\" (UID: \"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.170939 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.170905 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c5620b6-cae2-4cd1-a1c7-b798cb5951b8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-777496d5cd-grrtc\" (UID: \"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.174809 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.174771 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5c5620b6-cae2-4cd1-a1c7-b798cb5951b8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-777496d5cd-grrtc\" (UID: \"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.175190 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.175167 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5c5620b6-cae2-4cd1-a1c7-b798cb5951b8-ca\") pod \"cluster-proxy-proxy-agent-777496d5cd-grrtc\" (UID: \"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.176717 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.176672 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5c5620b6-cae2-4cd1-a1c7-b798cb5951b8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-777496d5cd-grrtc\" (UID: \"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.176920 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.176876 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5c5620b6-cae2-4cd1-a1c7-b798cb5951b8-hub\") pod \"cluster-proxy-proxy-agent-777496d5cd-grrtc\" (UID: \"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.185911 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.185887 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx467\" (UniqueName: \"kubernetes.io/projected/5c5620b6-cae2-4cd1-a1c7-b798cb5951b8-kube-api-access-hx467\") pod \"cluster-proxy-proxy-agent-777496d5cd-grrtc\" (UID: \"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.333762 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.333522 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" Jul 27 20:52:24.406222 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.406134 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wk44n" event={"ID":"668f5b4e-867c-47c2-b868-cb3f79c14f2a","Type":"ContainerStarted","Data":"2ed48df081af8fb3e34cea5f2e143d65a817ced8ee9b1dc274836b2f50097730"} Jul 27 20:52:24.435581 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:24.435361 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wk44n" podStartSLOduration=67.249014081 podStartE2EDuration="1m12.435337403s" totalImagesPullingTime="5.186323322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:51:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:52:24.433972229 +0000 UTC m=+72.957089952" watchObservedRunningTime="2026-07-27 20:52:24.435337403 +0000 UTC m=+72.958455128" Jul 27 20:52:25.067127 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:25.067063 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc"] Jul 27 20:52:25.072687 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:52:25.072657 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c5620b6_cae2_4cd1_a1c7_b798cb5951b8.slice/crio-ff57ce15664f3bd1926e8dec64db325761e9671d807aa45b06ec9c2a215fb5c7 WatchSource:0}: Error finding container ff57ce15664f3bd1926e8dec64db325761e9671d807aa45b06ec9c2a215fb5c7: Status 404 returned error can't find the container with id ff57ce15664f3bd1926e8dec64db325761e9671d807aa45b06ec9c2a215fb5c7 Jul 27 20:52:25.412374 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:25.412336 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j2k9d" event={"ID":"c1f55707-7360-492b-a75f-f2211a512fd3","Type":"ContainerStarted","Data":"ba9dcab2527e4fe7a4fbccdec83a4b0ba8bed39f5b7b4a82129d673c7423199e"} Jul 27 20:52:25.412846 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:25.412409 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j2k9d" event={"ID":"c1f55707-7360-492b-a75f-f2211a512fd3","Type":"ContainerStarted","Data":"80527a23f02b6d202d64e40c4f40467798667409e7fcdee080ece1791a139210"} Jul 27 20:52:25.412846 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:25.412445 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-j2k9d" Jul 27 20:52:25.413818 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:25.413791 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" event={"ID":"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8","Type":"ContainerStarted","Data":"ff57ce15664f3bd1926e8dec64db325761e9671d807aa45b06ec9c2a215fb5c7"} Jul 27 20:52:25.432709 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:25.432611 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-j2k9d" podStartSLOduration=36.718379782 podStartE2EDuration="38.432590577s" totalImagesPullingTime="1.714210795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:51:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:52:25.43089029 +0000 UTC m=+73.954008023" watchObservedRunningTime="2026-07-27 20:52:25.432590577 +0000 UTC m=+73.955708300" Jul 27 20:52:26.188570 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:26.188533 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9h98q\" (UID: \"9736f967-2f63-40e7-966e-628f22fe91d6\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9h98q" Jul 27 20:52:26.191403 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:26.191369 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9736f967-2f63-40e7-966e-628f22fe91d6-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9h98q\" (UID: \"9736f967-2f63-40e7-966e-628f22fe91d6\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9h98q" Jul 27 20:52:26.419718 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:26.419694 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9t85j\"" Jul 27 20:52:26.427688 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:26.427665 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9h98q" Jul 27 20:52:28.195714 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:28.195523 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-9h98q"] Jul 27 20:52:28.198455 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:52:28.198407 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9736f967_2f63_40e7_966e_628f22fe91d6.slice/crio-c074c98f6af2608ac7a39741a1503b6590ee54ea563cd6e6d0b07d96ed5bbf4b WatchSource:0}: Error finding container c074c98f6af2608ac7a39741a1503b6590ee54ea563cd6e6d0b07d96ed5bbf4b: Status 404 returned error can't find the container with id c074c98f6af2608ac7a39741a1503b6590ee54ea563cd6e6d0b07d96ed5bbf4b Jul 27 20:52:28.425511 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:28.425460 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9h98q" event={"ID":"9736f967-2f63-40e7-966e-628f22fe91d6","Type":"ContainerStarted","Data":"c074c98f6af2608ac7a39741a1503b6590ee54ea563cd6e6d0b07d96ed5bbf4b"} Jul 27 20:52:28.427333 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:28.427304 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4bhvm" event={"ID":"f7db41ad-fec7-4420-93d8-869d4ee1965b","Type":"ContainerStarted","Data":"0b5839dc65810fd2c45d4eb2ffeccc68684746da4a2b74e5a05d6b6971d33e64"} Jul 27 20:52:32.444686 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:32.444660 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4bhvm" event={"ID":"f7db41ad-fec7-4420-93d8-869d4ee1965b","Type":"ContainerStarted","Data":"24f3489409c54d4aff27a594677c551f8f945ec1ce1be9200737667d3b0c08f0"} Jul 27 20:52:32.446514 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:32.446487 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" event={"ID":"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8","Type":"ContainerStarted","Data":"d237ca19b63147b927caac0d923a4588ec6e8f115bd171f7f77e307a66fa5305"} Jul 27 20:52:32.446602 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:32.446535 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" event={"ID":"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8","Type":"ContainerStarted","Data":"cd70a14d631976783c4a5a264ee52f7b23752d8e66c08eed7b559924ec71fe39"} Jul 27 20:52:32.448044 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:32.448018 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9h98q" event={"ID":"9736f967-2f63-40e7-966e-628f22fe91d6","Type":"ContainerStarted","Data":"60c35611c07c2c2efb501afca43ce1e265bcdc7bb150c504cdfe136b7c10f739"} Jul 27 20:52:32.465726 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:32.465677 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4bhvm" podStartSLOduration=16.479375032 podStartE2EDuration="32.465660816s" totalImagesPullingTime="15.986285784s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:52:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:52:32.465056788 +0000 UTC m=+80.988174523" watchObservedRunningTime="2026-07-27 20:52:32.465660816 +0000 UTC m=+80.988778538" Jul 27 20:52:32.482281 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:32.482162 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9h98q" podStartSLOduration=34.381998149 podStartE2EDuration="38.482144384s" totalImagesPullingTime="4.100146235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:51:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:52:32.481907466 +0000 UTC m=+81.005025194" watchObservedRunningTime="2026-07-27 20:52:32.482144384 +0000 UTC m=+81.005262105" Jul 27 20:52:33.454643 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:33.454607 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" event={"ID":"5c5620b6-cae2-4cd1-a1c7-b798cb5951b8","Type":"ContainerStarted","Data":"b94e6003336bcba1fa4d728398011b0f918ed51504d1a63f51d9c57ff081a5c8"} Jul 27 20:52:33.479703 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:33.479648 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-777496d5cd-grrtc" podStartSLOduration=3.2555946909999998 podStartE2EDuration="10.479634021s" totalImagesPullingTime="7.22403933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:52:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:52:33.478661641 +0000 UTC m=+82.001779363" watchObservedRunningTime="2026-07-27 20:52:33.479634021 +0000 UTC m=+82.002751743" Jul 27 20:52:35.418852 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:35.418819 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-j2k9d" Jul 27 20:52:42.155725 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.155685 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lvtjr"] Jul 27 20:52:42.162017 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.161992 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.165996 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.165971 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 27 20:52:42.165996 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.165974 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 27 20:52:42.166301 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.165975 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 27 20:52:42.167423 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.167401 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wfrq9\"" Jul 27 20:52:42.167503 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.167406 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 27 20:52:42.167503 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.167439 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 27 20:52:42.167503 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.167412 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 27 20:52:42.318067 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.318030 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk89w\" (UniqueName: \"kubernetes.io/projected/11ab4a48-f264-4bb7-86ec-778b13da8825-kube-api-access-nk89w\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.318277 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.318077 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11ab4a48-f264-4bb7-86ec-778b13da8825-sys\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.318277 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.318104 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/11ab4a48-f264-4bb7-86ec-778b13da8825-root\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.318277 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.318123 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11ab4a48-f264-4bb7-86ec-778b13da8825-metrics-client-ca\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.318277 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.318159 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/11ab4a48-f264-4bb7-86ec-778b13da8825-node-exporter-accelerators-collector-config\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.318277 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.318184 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/11ab4a48-f264-4bb7-86ec-778b13da8825-node-exporter-tls\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.318277 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.318215 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/11ab4a48-f264-4bb7-86ec-778b13da8825-node-exporter-textfile\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.318277 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.318255 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11ab4a48-f264-4bb7-86ec-778b13da8825-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.318277 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.318279 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/11ab4a48-f264-4bb7-86ec-778b13da8825-node-exporter-wtmp\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.418618 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.418583 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11ab4a48-f264-4bb7-86ec-778b13da8825-sys\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.418793 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.418622 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/11ab4a48-f264-4bb7-86ec-778b13da8825-root\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.418793 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.418653 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11ab4a48-f264-4bb7-86ec-778b13da8825-metrics-client-ca\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.418793 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.418682 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/11ab4a48-f264-4bb7-86ec-778b13da8825-node-exporter-accelerators-collector-config\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.418793 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.418697 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11ab4a48-f264-4bb7-86ec-778b13da8825-sys\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.418793 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.418708 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/11ab4a48-f264-4bb7-86ec-778b13da8825-root\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.418793 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.418717 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/11ab4a48-f264-4bb7-86ec-778b13da8825-node-exporter-tls\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.418793 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.418770 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/11ab4a48-f264-4bb7-86ec-778b13da8825-node-exporter-textfile\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.419124 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.418796 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11ab4a48-f264-4bb7-86ec-778b13da8825-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.419124 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:42.418807 2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 27 20:52:42.419124 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.418836 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/11ab4a48-f264-4bb7-86ec-778b13da8825-node-exporter-wtmp\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.419124 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:52:42.418874 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11ab4a48-f264-4bb7-86ec-778b13da8825-node-exporter-tls podName:11ab4a48-f264-4bb7-86ec-778b13da8825 nodeName:}" failed. No retries permitted until 2026-07-27 20:52:42.918852679 +0000 UTC m=+91.441970384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/11ab4a48-f264-4bb7-86ec-778b13da8825-node-exporter-tls") pod "node-exporter-lvtjr" (UID: "11ab4a48-f264-4bb7-86ec-778b13da8825") : secret "node-exporter-tls" not found Jul 27 20:52:42.419124 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.418910 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk89w\" (UniqueName: \"kubernetes.io/projected/11ab4a48-f264-4bb7-86ec-778b13da8825-kube-api-access-nk89w\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.419124 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.418951 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/11ab4a48-f264-4bb7-86ec-778b13da8825-node-exporter-wtmp\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.419534 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.419151 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/11ab4a48-f264-4bb7-86ec-778b13da8825-node-exporter-textfile\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.419534 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.419309 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/11ab4a48-f264-4bb7-86ec-778b13da8825-node-exporter-accelerators-collector-config\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.419534 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.419430 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11ab4a48-f264-4bb7-86ec-778b13da8825-metrics-client-ca\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.421474 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.421449 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11ab4a48-f264-4bb7-86ec-778b13da8825-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.433109 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.433077 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk89w\" (UniqueName: \"kubernetes.io/projected/11ab4a48-f264-4bb7-86ec-778b13da8825-kube-api-access-nk89w\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.924034 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.923999 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/11ab4a48-f264-4bb7-86ec-778b13da8825-node-exporter-tls\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:42.926457 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:42.926439 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/11ab4a48-f264-4bb7-86ec-778b13da8825-node-exporter-tls\") pod \"node-exporter-lvtjr\" (UID: \"11ab4a48-f264-4bb7-86ec-778b13da8825\") " pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:43.071099 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:43.071070 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lvtjr" Jul 27 20:52:43.083707 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:52:43.083668 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11ab4a48_f264_4bb7_86ec_778b13da8825.slice/crio-e059eb94d66d8e358cd9a2d7bcd830b1916bb76e4728247f449c0d2fd458dae6 WatchSource:0}: Error finding container e059eb94d66d8e358cd9a2d7bcd830b1916bb76e4728247f449c0d2fd458dae6: Status 404 returned error can't find the container with id e059eb94d66d8e358cd9a2d7bcd830b1916bb76e4728247f449c0d2fd458dae6 Jul 27 20:52:43.490279 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:43.490223 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lvtjr" event={"ID":"11ab4a48-f264-4bb7-86ec-778b13da8825","Type":"ContainerStarted","Data":"e059eb94d66d8e358cd9a2d7bcd830b1916bb76e4728247f449c0d2fd458dae6"} Jul 27 20:52:44.411445 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:44.411367 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:52:44.495672 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:44.495641 2565 generic.go:356] "Generic (PLEG): container finished" podID="11ab4a48-f264-4bb7-86ec-778b13da8825" containerID="6c0788743e02042da9b16e645492cda54b6f5b656e6e5c56cef07d1ec784886c" exitCode=0 Jul 27 20:52:44.496031 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:44.495694 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lvtjr" event={"ID":"11ab4a48-f264-4bb7-86ec-778b13da8825","Type":"ContainerDied","Data":"6c0788743e02042da9b16e645492cda54b6f5b656e6e5c56cef07d1ec784886c"} Jul 27 20:52:45.501525 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:45.501490 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lvtjr" event={"ID":"11ab4a48-f264-4bb7-86ec-778b13da8825","Type":"ContainerStarted","Data":"15b0c4b7226d567e573be4f64cd61f996e0e4a02dcd48e4a879d7bf571e00d70"} Jul 27 20:52:45.501911 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:45.501573 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lvtjr" event={"ID":"11ab4a48-f264-4bb7-86ec-778b13da8825","Type":"ContainerStarted","Data":"3f26578f5306631ad86b8b9088931ee5f5fc970693f471ec709395d5ac34cbf5"} Jul 27 20:52:45.528670 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:45.528613 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lvtjr" podStartSLOduration=2.391150829 podStartE2EDuration="3.528597391s" totalImagesPullingTime="1.065293276s" totalInitContainerRuntime="72.153286ms" isStatefulPod=true podCreationTimestamp="2026-07-27 20:52:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:52:45.526073859 +0000 UTC m=+94.049191584" watchObservedRunningTime="2026-07-27 20:52:45.528597391 +0000 UTC m=+94.051715154" Jul 27 20:52:47.356325 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.356294 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s"] Jul 27 20:52:47.359620 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.359603 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.362579 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.362561 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 27 20:52:47.362691 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.362601 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 27 20:52:47.362940 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.362912 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 27 20:52:47.363019 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.362939 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 27 20:52:47.363019 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.362974 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 27 20:52:47.363934 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.363918 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-fhtm5\"" Jul 27 20:52:47.368843 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.368823 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 27 20:52:47.374044 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.374022 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s"] Jul 27 20:52:47.459981 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.459952 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aac785a0-88db-4144-ac71-bdd8b14388a5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.460139 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.459989 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/aac785a0-88db-4144-ac71-bdd8b14388a5-secret-telemeter-client\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.460139 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.460011 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aac785a0-88db-4144-ac71-bdd8b14388a5-serving-certs-ca-bundle\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.460139 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.460035 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74ncc\" (UniqueName: \"kubernetes.io/projected/aac785a0-88db-4144-ac71-bdd8b14388a5-kube-api-access-74ncc\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.460139 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.460057 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/aac785a0-88db-4144-ac71-bdd8b14388a5-federate-client-tls\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.460139 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.460085 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/aac785a0-88db-4144-ac71-bdd8b14388a5-telemeter-client-tls\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.460315 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.460162 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aac785a0-88db-4144-ac71-bdd8b14388a5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.460315 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.460193 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aac785a0-88db-4144-ac71-bdd8b14388a5-metrics-client-ca\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.560878 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.560840 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aac785a0-88db-4144-ac71-bdd8b14388a5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.560878 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.560881 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/aac785a0-88db-4144-ac71-bdd8b14388a5-secret-telemeter-client\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.561134 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.560902 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aac785a0-88db-4144-ac71-bdd8b14388a5-serving-certs-ca-bundle\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.561134 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.560922 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74ncc\" (UniqueName: \"kubernetes.io/projected/aac785a0-88db-4144-ac71-bdd8b14388a5-kube-api-access-74ncc\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.561134 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.560943 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/aac785a0-88db-4144-ac71-bdd8b14388a5-federate-client-tls\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.561134 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.560998 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/aac785a0-88db-4144-ac71-bdd8b14388a5-telemeter-client-tls\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.561134 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.561048 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aac785a0-88db-4144-ac71-bdd8b14388a5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.561134 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.561074 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aac785a0-88db-4144-ac71-bdd8b14388a5-metrics-client-ca\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.561813 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.561781 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aac785a0-88db-4144-ac71-bdd8b14388a5-metrics-client-ca\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.562011 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.561915 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aac785a0-88db-4144-ac71-bdd8b14388a5-serving-certs-ca-bundle\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.562011 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.562004 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aac785a0-88db-4144-ac71-bdd8b14388a5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.563606 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.563585 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/aac785a0-88db-4144-ac71-bdd8b14388a5-federate-client-tls\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.563702 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.563633 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/aac785a0-88db-4144-ac71-bdd8b14388a5-telemeter-client-tls\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.563992 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.563974 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aac785a0-88db-4144-ac71-bdd8b14388a5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.564174 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.564154 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/aac785a0-88db-4144-ac71-bdd8b14388a5-secret-telemeter-client\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.579992 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.579961 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74ncc\" (UniqueName: \"kubernetes.io/projected/aac785a0-88db-4144-ac71-bdd8b14388a5-kube-api-access-74ncc\") pod \"telemeter-client-6bdd54d9bb-kjr8s\" (UID: \"aac785a0-88db-4144-ac71-bdd8b14388a5\") " pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.670004 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.669972 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" Jul 27 20:52:47.822149 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:47.822062 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s"] Jul 27 20:52:47.825073 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:52:47.825051 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaac785a0_88db_4144_ac71_bdd8b14388a5.slice/crio-5e1241dad2251f6dca070f4ebdd76d6cbc51e4a054c8dba78d94700c17bfe369 WatchSource:0}: Error finding container 5e1241dad2251f6dca070f4ebdd76d6cbc51e4a054c8dba78d94700c17bfe369: Status 404 returned error can't find the container with id 5e1241dad2251f6dca070f4ebdd76d6cbc51e4a054c8dba78d94700c17bfe369 Jul 27 20:52:48.513871 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:48.513830 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" event={"ID":"aac785a0-88db-4144-ac71-bdd8b14388a5","Type":"ContainerStarted","Data":"5e1241dad2251f6dca070f4ebdd76d6cbc51e4a054c8dba78d94700c17bfe369"} Jul 27 20:52:49.374126 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:49.374094 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zt78l" Jul 27 20:52:50.524494 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:50.524446 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" event={"ID":"aac785a0-88db-4144-ac71-bdd8b14388a5","Type":"ContainerStarted","Data":"fb84629e8276e06e7535a34225abc87653b300ee08cdc23ce021d027137b69b9"} Jul 27 20:52:51.529943 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:51.529911 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" event={"ID":"aac785a0-88db-4144-ac71-bdd8b14388a5","Type":"ContainerStarted","Data":"434b265d0ae49fe255ed7160871aff4f6f0fef1f2f5af6350984a470dd2d0c5a"} Jul 27 20:52:51.530326 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:51.529975 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" event={"ID":"aac785a0-88db-4144-ac71-bdd8b14388a5","Type":"ContainerStarted","Data":"8329a465ea58ee17cc06e9eb461d7679a29063e4991aba57dc444d17b6e9ac97"} Jul 27 20:52:51.563936 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:51.563890 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6bdd54d9bb-kjr8s" podStartSLOduration=1.3640517349999999 podStartE2EDuration="4.563875579s" totalImagesPullingTime="3.199823844s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:52:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:52:51.562700139 +0000 UTC m=+100.085817860" watchObservedRunningTime="2026-07-27 20:52:51.563875579 +0000 UTC m=+100.086993300" Jul 27 20:52:51.699104 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:52:51.699064 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7655cf7f49-ldmrw"] Jul 27 20:53:05.641480 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:05.641446 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5psdr_78365e64-dcf0-450f-ae1e-d8dc161e3657/serve-healthcheck-canary/0.log" Jul 27 20:53:16.718633 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:16.718570 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" podUID="43cb7481-cfd4-4480-8523-3b2907b10931" containerName="registry" containerID="cri-o://4c88354f8a7e80bbd119f6245239d3e947b5a38863835f732ac4d4cfc6125080" gracePeriod=30 Jul 27 20:53:16.970625 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:16.970565 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:53:16.998351 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:16.989875 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/43cb7481-cfd4-4480-8523-3b2907b10931-registry-certificates\") pod \"43cb7481-cfd4-4480-8523-3b2907b10931\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " Jul 27 20:53:16.998351 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:16.989924 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-bound-sa-token\") pod \"43cb7481-cfd4-4480-8523-3b2907b10931\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " Jul 27 20:53:16.998351 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:16.989970 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/43cb7481-cfd4-4480-8523-3b2907b10931-ca-trust-extracted\") pod \"43cb7481-cfd4-4480-8523-3b2907b10931\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " Jul 27 20:53:16.998351 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:16.990023 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/43cb7481-cfd4-4480-8523-3b2907b10931-image-registry-private-configuration\") pod \"43cb7481-cfd4-4480-8523-3b2907b10931\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " Jul 27 20:53:16.998351 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:16.990058 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/43cb7481-cfd4-4480-8523-3b2907b10931-trusted-ca\") pod \"43cb7481-cfd4-4480-8523-3b2907b10931\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " Jul 27 20:53:16.998351 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:16.990098 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls\") pod \"43cb7481-cfd4-4480-8523-3b2907b10931\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " Jul 27 20:53:16.998351 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:16.990126 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vc589\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-kube-api-access-vc589\") pod \"43cb7481-cfd4-4480-8523-3b2907b10931\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " Jul 27 20:53:16.998351 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:16.990163 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/43cb7481-cfd4-4480-8523-3b2907b10931-installation-pull-secrets\") pod \"43cb7481-cfd4-4480-8523-3b2907b10931\" (UID: \"43cb7481-cfd4-4480-8523-3b2907b10931\") " Jul 27 20:53:16.998351 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:16.994477 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43cb7481-cfd4-4480-8523-3b2907b10931-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "43cb7481-cfd4-4480-8523-3b2907b10931" (UID: "43cb7481-cfd4-4480-8523-3b2907b10931"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 20:53:16.998351 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:16.994806 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43cb7481-cfd4-4480-8523-3b2907b10931-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "43cb7481-cfd4-4480-8523-3b2907b10931" (UID: "43cb7481-cfd4-4480-8523-3b2907b10931"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 20:53:16.998351 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:16.996104 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43cb7481-cfd4-4480-8523-3b2907b10931-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "43cb7481-cfd4-4480-8523-3b2907b10931" (UID: "43cb7481-cfd4-4480-8523-3b2907b10931"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 20:53:16.998351 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:16.996440 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43cb7481-cfd4-4480-8523-3b2907b10931-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "43cb7481-cfd4-4480-8523-3b2907b10931" (UID: "43cb7481-cfd4-4480-8523-3b2907b10931"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 20:53:17.000663 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.000623 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "43cb7481-cfd4-4480-8523-3b2907b10931" (UID: "43cb7481-cfd4-4480-8523-3b2907b10931"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 20:53:17.001650 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.001622 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "43cb7481-cfd4-4480-8523-3b2907b10931" (UID: "43cb7481-cfd4-4480-8523-3b2907b10931"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 20:53:17.001884 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.001861 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-kube-api-access-vc589" (OuterVolumeSpecName: "kube-api-access-vc589") pod "43cb7481-cfd4-4480-8523-3b2907b10931" (UID: "43cb7481-cfd4-4480-8523-3b2907b10931"). InnerVolumeSpecName "kube-api-access-vc589". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 20:53:17.009266 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.009201 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43cb7481-cfd4-4480-8523-3b2907b10931-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "43cb7481-cfd4-4480-8523-3b2907b10931" (UID: "43cb7481-cfd4-4480-8523-3b2907b10931"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 20:53:17.091454 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.091419 2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/43cb7481-cfd4-4480-8523-3b2907b10931-image-registry-private-configuration\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:53:17.091454 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.091453 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/43cb7481-cfd4-4480-8523-3b2907b10931-trusted-ca\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:53:17.091454 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.091463 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-registry-tls\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:53:17.091666 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.091472 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vc589\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-kube-api-access-vc589\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:53:17.091666 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.091480 2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/43cb7481-cfd4-4480-8523-3b2907b10931-installation-pull-secrets\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:53:17.091666 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.091489 2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/43cb7481-cfd4-4480-8523-3b2907b10931-registry-certificates\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:53:17.091666 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.091497 2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/43cb7481-cfd4-4480-8523-3b2907b10931-bound-sa-token\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:53:17.091666 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.091506 2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/43cb7481-cfd4-4480-8523-3b2907b10931-ca-trust-extracted\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:53:17.621328 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.621284 2565 generic.go:356] "Generic (PLEG): container finished" podID="43cb7481-cfd4-4480-8523-3b2907b10931" containerID="4c88354f8a7e80bbd119f6245239d3e947b5a38863835f732ac4d4cfc6125080" exitCode=0 Jul 27 20:53:17.621328 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.621333 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" Jul 27 20:53:17.621523 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.621363 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" event={"ID":"43cb7481-cfd4-4480-8523-3b2907b10931","Type":"ContainerDied","Data":"4c88354f8a7e80bbd119f6245239d3e947b5a38863835f732ac4d4cfc6125080"} Jul 27 20:53:17.621523 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.621422 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7655cf7f49-ldmrw" event={"ID":"43cb7481-cfd4-4480-8523-3b2907b10931","Type":"ContainerDied","Data":"cf0e0f69202b4c3be78d5c8d770d00544a1f6ce27e92fc749a3d8f3b2e972f78"} Jul 27 20:53:17.621523 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.621444 2565 scope.go:117] "RemoveContainer" containerID="4c88354f8a7e80bbd119f6245239d3e947b5a38863835f732ac4d4cfc6125080" Jul 27 20:53:17.631061 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.631038 2565 scope.go:117] "RemoveContainer" containerID="4c88354f8a7e80bbd119f6245239d3e947b5a38863835f732ac4d4cfc6125080" Jul 27 20:53:17.631475 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:53:17.631443 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c88354f8a7e80bbd119f6245239d3e947b5a38863835f732ac4d4cfc6125080\": container with ID starting with 4c88354f8a7e80bbd119f6245239d3e947b5a38863835f732ac4d4cfc6125080 not found: ID does not exist" containerID="4c88354f8a7e80bbd119f6245239d3e947b5a38863835f732ac4d4cfc6125080" Jul 27 20:53:17.631572 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.631485 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c88354f8a7e80bbd119f6245239d3e947b5a38863835f732ac4d4cfc6125080"} err="failed to get container status \"4c88354f8a7e80bbd119f6245239d3e947b5a38863835f732ac4d4cfc6125080\": rpc error: code = NotFound desc = could not find container \"4c88354f8a7e80bbd119f6245239d3e947b5a38863835f732ac4d4cfc6125080\": container with ID starting with 4c88354f8a7e80bbd119f6245239d3e947b5a38863835f732ac4d4cfc6125080 not found: ID does not exist" Jul 27 20:53:17.642837 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.642815 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7655cf7f49-ldmrw"] Jul 27 20:53:17.649059 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:17.649038 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7655cf7f49-ldmrw"] Jul 27 20:53:18.052061 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:18.052030 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43cb7481-cfd4-4480-8523-3b2907b10931" path="/var/lib/kubelet/pods/43cb7481-cfd4-4480-8523-3b2907b10931/volumes" Jul 27 20:53:35.683190 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:35.683154 2565 generic.go:356] "Generic (PLEG): container finished" podID="3f79a474-2b0c-4996-876d-ab959ea1f2d9" containerID="82a37d920b4c8f7452df0ecc64684260c8d54a7494d5ca6db07b15932159026d" exitCode=0 Jul 27 20:53:35.683651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:35.683247 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-9chx2" event={"ID":"3f79a474-2b0c-4996-876d-ab959ea1f2d9","Type":"ContainerDied","Data":"82a37d920b4c8f7452df0ecc64684260c8d54a7494d5ca6db07b15932159026d"} Jul 27 20:53:35.683651 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:35.683561 2565 scope.go:117] "RemoveContainer" containerID="82a37d920b4c8f7452df0ecc64684260c8d54a7494d5ca6db07b15932159026d" Jul 27 20:53:36.688860 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:53:36.688827 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-9chx2" event={"ID":"3f79a474-2b0c-4996-876d-ab959ea1f2d9","Type":"ContainerStarted","Data":"044c8624781be8fc0afe44918c0fe2dfecf24c8e4e9d5b6cc08c7db2769524d2"} Jul 27 20:54:41.960703 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:41.960666 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z"] Jul 27 20:54:41.961166 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:41.960958 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43cb7481-cfd4-4480-8523-3b2907b10931" containerName="registry" Jul 27 20:54:41.961166 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:41.960969 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="43cb7481-cfd4-4480-8523-3b2907b10931" containerName="registry" Jul 27 20:54:41.961166 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:41.961022 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="43cb7481-cfd4-4480-8523-3b2907b10931" containerName="registry" Jul 27 20:54:41.963457 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:41.963435 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z" Jul 27 20:54:41.966300 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:41.966278 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ckf8j\"" Jul 27 20:54:41.966300 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:41.966291 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 27 20:54:41.966460 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:41.966308 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 27 20:54:41.973879 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:41.973854 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z"] Jul 27 20:54:42.098993 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:42.098960 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c335dd20-b6d4-41c5-baed-60e39acbc2db-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z\" (UID: \"c335dd20-b6d4-41c5-baed-60e39acbc2db\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z" Jul 27 20:54:42.098993 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:42.099007 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c335dd20-b6d4-41c5-baed-60e39acbc2db-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z\" (UID: \"c335dd20-b6d4-41c5-baed-60e39acbc2db\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z" Jul 27 20:54:42.099208 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:42.099038 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkcfk\" (UniqueName: \"kubernetes.io/projected/c335dd20-b6d4-41c5-baed-60e39acbc2db-kube-api-access-qkcfk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z\" (UID: \"c335dd20-b6d4-41c5-baed-60e39acbc2db\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z" Jul 27 20:54:42.200356 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:42.200319 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c335dd20-b6d4-41c5-baed-60e39acbc2db-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z\" (UID: \"c335dd20-b6d4-41c5-baed-60e39acbc2db\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z" Jul 27 20:54:42.200516 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:42.200379 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c335dd20-b6d4-41c5-baed-60e39acbc2db-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z\" (UID: \"c335dd20-b6d4-41c5-baed-60e39acbc2db\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z" Jul 27 20:54:42.200516 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:42.200423 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkcfk\" (UniqueName: \"kubernetes.io/projected/c335dd20-b6d4-41c5-baed-60e39acbc2db-kube-api-access-qkcfk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z\" (UID: \"c335dd20-b6d4-41c5-baed-60e39acbc2db\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z" Jul 27 20:54:42.200802 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:42.200779 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c335dd20-b6d4-41c5-baed-60e39acbc2db-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z\" (UID: \"c335dd20-b6d4-41c5-baed-60e39acbc2db\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z" Jul 27 20:54:42.200889 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:42.200818 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c335dd20-b6d4-41c5-baed-60e39acbc2db-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z\" (UID: \"c335dd20-b6d4-41c5-baed-60e39acbc2db\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z" Jul 27 20:54:42.209721 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:42.209695 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkcfk\" (UniqueName: \"kubernetes.io/projected/c335dd20-b6d4-41c5-baed-60e39acbc2db-kube-api-access-qkcfk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z\" (UID: \"c335dd20-b6d4-41c5-baed-60e39acbc2db\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z" Jul 27 20:54:42.272597 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:42.272521 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z" Jul 27 20:54:42.414439 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:42.414406 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z"] Jul 27 20:54:42.418290 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:54:42.418260 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc335dd20_b6d4_41c5_baed_60e39acbc2db.slice/crio-c2f39009a5b750dce9ed0596b6ecc9655d109b7eb74fa94cd4507c837e4db917 WatchSource:0}: Error finding container c2f39009a5b750dce9ed0596b6ecc9655d109b7eb74fa94cd4507c837e4db917: Status 404 returned error can't find the container with id c2f39009a5b750dce9ed0596b6ecc9655d109b7eb74fa94cd4507c837e4db917 Jul 27 20:54:42.909534 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:42.909493 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z" event={"ID":"c335dd20-b6d4-41c5-baed-60e39acbc2db","Type":"ContainerStarted","Data":"c2f39009a5b750dce9ed0596b6ecc9655d109b7eb74fa94cd4507c837e4db917"} Jul 27 20:54:48.931815 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:48.931781 2565 generic.go:356] "Generic (PLEG): container finished" podID="c335dd20-b6d4-41c5-baed-60e39acbc2db" containerID="16239036ee68bd5250dcee19e6930d9e33b36f8a1228777c51a9d2c43816cb4c" exitCode=0 Jul 27 20:54:48.932180 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:48.931836 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z" event={"ID":"c335dd20-b6d4-41c5-baed-60e39acbc2db","Type":"ContainerDied","Data":"16239036ee68bd5250dcee19e6930d9e33b36f8a1228777c51a9d2c43816cb4c"} Jul 27 20:54:51.945294 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:51.945255 2565 generic.go:356] "Generic (PLEG): container finished" podID="c335dd20-b6d4-41c5-baed-60e39acbc2db" containerID="40e1a3d245fdf98eb78ba73d3270ccd90bef24fc64c5ae5a4ce6244b9944769e" exitCode=0 Jul 27 20:54:51.945752 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:51.945303 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z" event={"ID":"c335dd20-b6d4-41c5-baed-60e39acbc2db","Type":"ContainerDied","Data":"40e1a3d245fdf98eb78ba73d3270ccd90bef24fc64c5ae5a4ce6244b9944769e"} Jul 27 20:54:59.977400 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:59.977357 2565 generic.go:356] "Generic (PLEG): container finished" podID="c335dd20-b6d4-41c5-baed-60e39acbc2db" containerID="e1f95172992b7a007b2acc0791a2494c33174dca1780dc0a66e1493d584bfcf9" exitCode=0 Jul 27 20:54:59.977770 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:54:59.977424 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z" event={"ID":"c335dd20-b6d4-41c5-baed-60e39acbc2db","Type":"ContainerDied","Data":"e1f95172992b7a007b2acc0791a2494c33174dca1780dc0a66e1493d584bfcf9"} Jul 27 20:55:01.110680 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:01.110651 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z" Jul 27 20:55:01.139453 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:01.139419 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c335dd20-b6d4-41c5-baed-60e39acbc2db-bundle\") pod \"c335dd20-b6d4-41c5-baed-60e39acbc2db\" (UID: \"c335dd20-b6d4-41c5-baed-60e39acbc2db\") " Jul 27 20:55:01.139605 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:01.139519 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkcfk\" (UniqueName: \"kubernetes.io/projected/c335dd20-b6d4-41c5-baed-60e39acbc2db-kube-api-access-qkcfk\") pod \"c335dd20-b6d4-41c5-baed-60e39acbc2db\" (UID: \"c335dd20-b6d4-41c5-baed-60e39acbc2db\") " Jul 27 20:55:01.139605 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:01.139555 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c335dd20-b6d4-41c5-baed-60e39acbc2db-util\") pod \"c335dd20-b6d4-41c5-baed-60e39acbc2db\" (UID: \"c335dd20-b6d4-41c5-baed-60e39acbc2db\") " Jul 27 20:55:01.140296 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:01.140261 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c335dd20-b6d4-41c5-baed-60e39acbc2db-bundle" (OuterVolumeSpecName: "bundle") pod "c335dd20-b6d4-41c5-baed-60e39acbc2db" (UID: "c335dd20-b6d4-41c5-baed-60e39acbc2db"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 20:55:01.141902 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:01.141873 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c335dd20-b6d4-41c5-baed-60e39acbc2db-kube-api-access-qkcfk" (OuterVolumeSpecName: "kube-api-access-qkcfk") pod "c335dd20-b6d4-41c5-baed-60e39acbc2db" (UID: "c335dd20-b6d4-41c5-baed-60e39acbc2db"). InnerVolumeSpecName "kube-api-access-qkcfk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 20:55:01.144155 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:01.144134 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c335dd20-b6d4-41c5-baed-60e39acbc2db-util" (OuterVolumeSpecName: "util") pod "c335dd20-b6d4-41c5-baed-60e39acbc2db" (UID: "c335dd20-b6d4-41c5-baed-60e39acbc2db"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 20:55:01.241222 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:01.241108 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c335dd20-b6d4-41c5-baed-60e39acbc2db-util\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:55:01.241222 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:01.241144 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c335dd20-b6d4-41c5-baed-60e39acbc2db-bundle\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:55:01.241222 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:01.241159 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkcfk\" (UniqueName: \"kubernetes.io/projected/c335dd20-b6d4-41c5-baed-60e39acbc2db-kube-api-access-qkcfk\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:55:01.986346 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:01.986313 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z" event={"ID":"c335dd20-b6d4-41c5-baed-60e39acbc2db","Type":"ContainerDied","Data":"c2f39009a5b750dce9ed0596b6ecc9655d109b7eb74fa94cd4507c837e4db917"} Jul 27 20:55:01.986505 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:01.986358 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2f39009a5b750dce9ed0596b6ecc9655d109b7eb74fa94cd4507c837e4db917" Jul 27 20:55:01.986505 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:01.986365 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402jx2z" Jul 27 20:55:19.294218 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.294185 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-h5bg4"] Jul 27 20:55:19.294700 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.294453 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c335dd20-b6d4-41c5-baed-60e39acbc2db" containerName="extract" Jul 27 20:55:19.294700 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.294464 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c335dd20-b6d4-41c5-baed-60e39acbc2db" containerName="extract" Jul 27 20:55:19.294700 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.294476 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c335dd20-b6d4-41c5-baed-60e39acbc2db" containerName="util" Jul 27 20:55:19.294700 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.294482 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c335dd20-b6d4-41c5-baed-60e39acbc2db" containerName="util" Jul 27 20:55:19.294700 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.294489 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c335dd20-b6d4-41c5-baed-60e39acbc2db" containerName="pull" Jul 27 20:55:19.294700 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.294494 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c335dd20-b6d4-41c5-baed-60e39acbc2db" containerName="pull" Jul 27 20:55:19.294700 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.294532 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c335dd20-b6d4-41c5-baed-60e39acbc2db" containerName="extract" Jul 27 20:55:19.298989 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.298973 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-h5bg4" Jul 27 20:55:19.302575 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.302531 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 27 20:55:19.302575 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.302572 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 27 20:55:19.302820 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.302622 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-x55sf\"" Jul 27 20:55:19.315672 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.315647 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-h5bg4"] Jul 27 20:55:19.367653 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.367617 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkx9h\" (UniqueName: \"kubernetes.io/projected/5b8e5eb4-fed6-4533-9f60-4e4412c6c667-kube-api-access-fkx9h\") pod \"cert-manager-55b4d954d5-h5bg4\" (UID: \"5b8e5eb4-fed6-4533-9f60-4e4412c6c667\") " pod="cert-manager/cert-manager-55b4d954d5-h5bg4" Jul 27 20:55:19.367778 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.367655 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5b8e5eb4-fed6-4533-9f60-4e4412c6c667-bound-sa-token\") pod \"cert-manager-55b4d954d5-h5bg4\" (UID: \"5b8e5eb4-fed6-4533-9f60-4e4412c6c667\") " pod="cert-manager/cert-manager-55b4d954d5-h5bg4" Jul 27 20:55:19.468259 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.468203 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkx9h\" (UniqueName: \"kubernetes.io/projected/5b8e5eb4-fed6-4533-9f60-4e4412c6c667-kube-api-access-fkx9h\") pod \"cert-manager-55b4d954d5-h5bg4\" (UID: \"5b8e5eb4-fed6-4533-9f60-4e4412c6c667\") " pod="cert-manager/cert-manager-55b4d954d5-h5bg4" Jul 27 20:55:19.468259 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.468259 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5b8e5eb4-fed6-4533-9f60-4e4412c6c667-bound-sa-token\") pod \"cert-manager-55b4d954d5-h5bg4\" (UID: \"5b8e5eb4-fed6-4533-9f60-4e4412c6c667\") " pod="cert-manager/cert-manager-55b4d954d5-h5bg4" Jul 27 20:55:19.481210 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.481179 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkx9h\" (UniqueName: \"kubernetes.io/projected/5b8e5eb4-fed6-4533-9f60-4e4412c6c667-kube-api-access-fkx9h\") pod \"cert-manager-55b4d954d5-h5bg4\" (UID: \"5b8e5eb4-fed6-4533-9f60-4e4412c6c667\") " pod="cert-manager/cert-manager-55b4d954d5-h5bg4" Jul 27 20:55:19.482706 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.482689 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5b8e5eb4-fed6-4533-9f60-4e4412c6c667-bound-sa-token\") pod \"cert-manager-55b4d954d5-h5bg4\" (UID: \"5b8e5eb4-fed6-4533-9f60-4e4412c6c667\") " pod="cert-manager/cert-manager-55b4d954d5-h5bg4" Jul 27 20:55:19.609073 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.608956 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-h5bg4" Jul 27 20:55:19.744359 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:19.744334 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-h5bg4"] Jul 27 20:55:19.746903 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:55:19.746880 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b8e5eb4_fed6_4533_9f60_4e4412c6c667.slice/crio-6642044f9d0e255d3a130ed7a915ba25d71d1782203a8002b37717d22f3fe23d WatchSource:0}: Error finding container 6642044f9d0e255d3a130ed7a915ba25d71d1782203a8002b37717d22f3fe23d: Status 404 returned error can't find the container with id 6642044f9d0e255d3a130ed7a915ba25d71d1782203a8002b37717d22f3fe23d Jul 27 20:55:20.050796 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:20.050765 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-h5bg4" event={"ID":"5b8e5eb4-fed6-4533-9f60-4e4412c6c667","Type":"ContainerStarted","Data":"6642044f9d0e255d3a130ed7a915ba25d71d1782203a8002b37717d22f3fe23d"} Jul 27 20:55:23.063884 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:23.063843 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-h5bg4" event={"ID":"5b8e5eb4-fed6-4533-9f60-4e4412c6c667","Type":"ContainerStarted","Data":"bfaba499ed8962d15c5a093e9ee5d40b2a4af77809abfd2d670df5a9cd368bb4"} Jul 27 20:55:23.089702 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:23.089656 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-h5bg4" podStartSLOduration=1.492956734 podStartE2EDuration="4.089640973s" totalImagesPullingTime="2.596684239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:55:23.089260886 +0000 UTC m=+251.612378601" watchObservedRunningTime="2026-07-27 20:55:23.089640973 +0000 UTC m=+251.612758695" Jul 27 20:55:33.041771 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:33.041682 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cmhhb"] Jul 27 20:55:33.043932 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:33.043911 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cmhhb" Jul 27 20:55:33.046661 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:33.046633 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 27 20:55:33.046769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:33.046713 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 27 20:55:33.048108 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:33.048064 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-m64dn\"" Jul 27 20:55:33.053398 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:33.053375 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cmhhb"] Jul 27 20:55:33.170878 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:33.170837 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw6tr\" (UniqueName: \"kubernetes.io/projected/4b7d257f-94bb-4b12-9cdc-7344915e51d7-kube-api-access-cw6tr\") pod \"openshift-lws-operator-fd8ccff4c-cmhhb\" (UID: \"4b7d257f-94bb-4b12-9cdc-7344915e51d7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cmhhb" Jul 27 20:55:33.171048 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:33.170925 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b7d257f-94bb-4b12-9cdc-7344915e51d7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cmhhb\" (UID: \"4b7d257f-94bb-4b12-9cdc-7344915e51d7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cmhhb" Jul 27 20:55:33.272061 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:33.272020 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b7d257f-94bb-4b12-9cdc-7344915e51d7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cmhhb\" (UID: \"4b7d257f-94bb-4b12-9cdc-7344915e51d7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cmhhb" Jul 27 20:55:33.272293 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:33.272076 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw6tr\" (UniqueName: \"kubernetes.io/projected/4b7d257f-94bb-4b12-9cdc-7344915e51d7-kube-api-access-cw6tr\") pod \"openshift-lws-operator-fd8ccff4c-cmhhb\" (UID: \"4b7d257f-94bb-4b12-9cdc-7344915e51d7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cmhhb" Jul 27 20:55:33.272467 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:33.272432 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b7d257f-94bb-4b12-9cdc-7344915e51d7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cmhhb\" (UID: \"4b7d257f-94bb-4b12-9cdc-7344915e51d7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cmhhb" Jul 27 20:55:33.281815 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:33.281783 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw6tr\" (UniqueName: \"kubernetes.io/projected/4b7d257f-94bb-4b12-9cdc-7344915e51d7-kube-api-access-cw6tr\") pod \"openshift-lws-operator-fd8ccff4c-cmhhb\" (UID: \"4b7d257f-94bb-4b12-9cdc-7344915e51d7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cmhhb" Jul 27 20:55:33.353074 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:33.352994 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cmhhb" Jul 27 20:55:33.485383 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:33.485347 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cmhhb"] Jul 27 20:55:33.489575 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:55:33.489547 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b7d257f_94bb_4b12_9cdc_7344915e51d7.slice/crio-a7dbc01bc0933fd38993e52a35b2a2758cc280475cc4d5eba9601b45815206f3 WatchSource:0}: Error finding container a7dbc01bc0933fd38993e52a35b2a2758cc280475cc4d5eba9601b45815206f3: Status 404 returned error can't find the container with id a7dbc01bc0933fd38993e52a35b2a2758cc280475cc4d5eba9601b45815206f3 Jul 27 20:55:34.103292 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:34.103256 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cmhhb" event={"ID":"4b7d257f-94bb-4b12-9cdc-7344915e51d7","Type":"ContainerStarted","Data":"a7dbc01bc0933fd38993e52a35b2a2758cc280475cc4d5eba9601b45815206f3"} Jul 27 20:55:36.114136 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:36.114055 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cmhhb" event={"ID":"4b7d257f-94bb-4b12-9cdc-7344915e51d7","Type":"ContainerStarted","Data":"df2fea226a45e8118f8ed54ceafcdf6de252d8c50378bb51c44e10210364f51e"} Jul 27 20:55:36.130507 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:36.130460 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cmhhb" podStartSLOduration=0.770975648 podStartE2EDuration="3.130444208s" totalImagesPullingTime="2.35946856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:55:36.129529994 +0000 UTC m=+264.652647708" watchObservedRunningTime="2026-07-27 20:55:36.130444208 +0000 UTC m=+264.653561931" Jul 27 20:55:38.894607 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:38.894575 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct"] Jul 27 20:55:38.897306 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:38.897290 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct" Jul 27 20:55:38.900164 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:38.900140 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 27 20:55:38.900754 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:38.900738 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ckf8j\"" Jul 27 20:55:38.901507 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:38.901483 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 27 20:55:38.908699 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:38.908679 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct"] Jul 27 20:55:39.026354 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:39.026320 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e09d53c6-7715-4732-aafb-9a9b56a3f54e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct\" (UID: \"e09d53c6-7715-4732-aafb-9a9b56a3f54e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct" Jul 27 20:55:39.026510 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:39.026361 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6cm7\" (UniqueName: \"kubernetes.io/projected/e09d53c6-7715-4732-aafb-9a9b56a3f54e-kube-api-access-s6cm7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct\" (UID: \"e09d53c6-7715-4732-aafb-9a9b56a3f54e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct" Jul 27 20:55:39.026510 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:39.026397 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e09d53c6-7715-4732-aafb-9a9b56a3f54e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct\" (UID: \"e09d53c6-7715-4732-aafb-9a9b56a3f54e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct" Jul 27 20:55:39.127367 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:39.127340 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e09d53c6-7715-4732-aafb-9a9b56a3f54e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct\" (UID: \"e09d53c6-7715-4732-aafb-9a9b56a3f54e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct" Jul 27 20:55:39.127529 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:39.127391 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e09d53c6-7715-4732-aafb-9a9b56a3f54e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct\" (UID: \"e09d53c6-7715-4732-aafb-9a9b56a3f54e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct" Jul 27 20:55:39.127529 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:39.127507 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6cm7\" (UniqueName: \"kubernetes.io/projected/e09d53c6-7715-4732-aafb-9a9b56a3f54e-kube-api-access-s6cm7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct\" (UID: \"e09d53c6-7715-4732-aafb-9a9b56a3f54e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct" Jul 27 20:55:39.127711 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:39.127693 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e09d53c6-7715-4732-aafb-9a9b56a3f54e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct\" (UID: \"e09d53c6-7715-4732-aafb-9a9b56a3f54e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct" Jul 27 20:55:39.127779 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:39.127761 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e09d53c6-7715-4732-aafb-9a9b56a3f54e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct\" (UID: \"e09d53c6-7715-4732-aafb-9a9b56a3f54e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct" Jul 27 20:55:39.138221 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:39.138193 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6cm7\" (UniqueName: \"kubernetes.io/projected/e09d53c6-7715-4732-aafb-9a9b56a3f54e-kube-api-access-s6cm7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct\" (UID: \"e09d53c6-7715-4732-aafb-9a9b56a3f54e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct" Jul 27 20:55:39.205694 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:39.205658 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct" Jul 27 20:55:39.337365 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:39.337338 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct"] Jul 27 20:55:39.339996 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:55:39.339971 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode09d53c6_7715_4732_aafb_9a9b56a3f54e.slice/crio-a492d5c6fa37696525e1d3c83e6781622c713e94fcbcbdf8344e3a453a27709d WatchSource:0}: Error finding container a492d5c6fa37696525e1d3c83e6781622c713e94fcbcbdf8344e3a453a27709d: Status 404 returned error can't find the container with id a492d5c6fa37696525e1d3c83e6781622c713e94fcbcbdf8344e3a453a27709d Jul 27 20:55:40.129000 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:40.128969 2565 generic.go:356] "Generic (PLEG): container finished" podID="e09d53c6-7715-4732-aafb-9a9b56a3f54e" containerID="431762df9e91578d98cb6e5e2b22ee354148b42dbc1200eef66802313d554255" exitCode=0 Jul 27 20:55:40.129374 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:40.129017 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct" event={"ID":"e09d53c6-7715-4732-aafb-9a9b56a3f54e","Type":"ContainerDied","Data":"431762df9e91578d98cb6e5e2b22ee354148b42dbc1200eef66802313d554255"} Jul 27 20:55:40.129374 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:40.129050 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct" event={"ID":"e09d53c6-7715-4732-aafb-9a9b56a3f54e","Type":"ContainerStarted","Data":"a492d5c6fa37696525e1d3c83e6781622c713e94fcbcbdf8344e3a453a27709d"} Jul 27 20:55:41.134261 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:41.134171 2565 generic.go:356] "Generic (PLEG): container finished" podID="e09d53c6-7715-4732-aafb-9a9b56a3f54e" containerID="b807af265963cf8b5c2abf527548fa94ed22ceb5d7282550b56daff2b4c91b56" exitCode=0 Jul 27 20:55:41.134598 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:41.134268 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct" event={"ID":"e09d53c6-7715-4732-aafb-9a9b56a3f54e","Type":"ContainerDied","Data":"b807af265963cf8b5c2abf527548fa94ed22ceb5d7282550b56daff2b4c91b56"} Jul 27 20:55:42.139840 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:42.139808 2565 generic.go:356] "Generic (PLEG): container finished" podID="e09d53c6-7715-4732-aafb-9a9b56a3f54e" containerID="4c32d5018ce9ace404080ff49e33a575ae5ad524d233c165825bec074320e5dc" exitCode=0 Jul 27 20:55:42.140278 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:42.139849 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct" event={"ID":"e09d53c6-7715-4732-aafb-9a9b56a3f54e","Type":"ContainerDied","Data":"4c32d5018ce9ace404080ff49e33a575ae5ad524d233c165825bec074320e5dc"} Jul 27 20:55:43.263483 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:43.263456 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct" Jul 27 20:55:43.359856 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:43.359820 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e09d53c6-7715-4732-aafb-9a9b56a3f54e-bundle\") pod \"e09d53c6-7715-4732-aafb-9a9b56a3f54e\" (UID: \"e09d53c6-7715-4732-aafb-9a9b56a3f54e\") " Jul 27 20:55:43.360029 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:43.359912 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6cm7\" (UniqueName: \"kubernetes.io/projected/e09d53c6-7715-4732-aafb-9a9b56a3f54e-kube-api-access-s6cm7\") pod \"e09d53c6-7715-4732-aafb-9a9b56a3f54e\" (UID: \"e09d53c6-7715-4732-aafb-9a9b56a3f54e\") " Jul 27 20:55:43.360029 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:43.359950 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e09d53c6-7715-4732-aafb-9a9b56a3f54e-util\") pod \"e09d53c6-7715-4732-aafb-9a9b56a3f54e\" (UID: \"e09d53c6-7715-4732-aafb-9a9b56a3f54e\") " Jul 27 20:55:43.360697 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:43.360621 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e09d53c6-7715-4732-aafb-9a9b56a3f54e-bundle" (OuterVolumeSpecName: "bundle") pod "e09d53c6-7715-4732-aafb-9a9b56a3f54e" (UID: "e09d53c6-7715-4732-aafb-9a9b56a3f54e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 20:55:43.362163 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:43.362135 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e09d53c6-7715-4732-aafb-9a9b56a3f54e-kube-api-access-s6cm7" (OuterVolumeSpecName: "kube-api-access-s6cm7") pod "e09d53c6-7715-4732-aafb-9a9b56a3f54e" (UID: "e09d53c6-7715-4732-aafb-9a9b56a3f54e"). InnerVolumeSpecName "kube-api-access-s6cm7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 20:55:43.365657 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:43.365637 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e09d53c6-7715-4732-aafb-9a9b56a3f54e-util" (OuterVolumeSpecName: "util") pod "e09d53c6-7715-4732-aafb-9a9b56a3f54e" (UID: "e09d53c6-7715-4732-aafb-9a9b56a3f54e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 20:55:43.460834 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:43.460804 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e09d53c6-7715-4732-aafb-9a9b56a3f54e-util\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:55:43.460834 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:43.460829 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e09d53c6-7715-4732-aafb-9a9b56a3f54e-bundle\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:55:43.460834 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:43.460839 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6cm7\" (UniqueName: \"kubernetes.io/projected/e09d53c6-7715-4732-aafb-9a9b56a3f54e-kube-api-access-s6cm7\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:55:44.149718 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:44.149678 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct" event={"ID":"e09d53c6-7715-4732-aafb-9a9b56a3f54e","Type":"ContainerDied","Data":"a492d5c6fa37696525e1d3c83e6781622c713e94fcbcbdf8344e3a453a27709d"} Jul 27 20:55:44.149718 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:44.149700 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af98qct" Jul 27 20:55:44.149718 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:44.149724 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a492d5c6fa37696525e1d3c83e6781622c713e94fcbcbdf8344e3a453a27709d" Jul 27 20:55:53.710658 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.710622 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw"] Jul 27 20:55:53.711074 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.710869 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e09d53c6-7715-4732-aafb-9a9b56a3f54e" containerName="pull" Jul 27 20:55:53.711074 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.710881 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e09d53c6-7715-4732-aafb-9a9b56a3f54e" containerName="pull" Jul 27 20:55:53.711074 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.710895 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e09d53c6-7715-4732-aafb-9a9b56a3f54e" containerName="extract" Jul 27 20:55:53.711074 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.710900 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e09d53c6-7715-4732-aafb-9a9b56a3f54e" containerName="extract" Jul 27 20:55:53.711074 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.710912 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e09d53c6-7715-4732-aafb-9a9b56a3f54e" containerName="util" Jul 27 20:55:53.711074 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.710918 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e09d53c6-7715-4732-aafb-9a9b56a3f54e" containerName="util" Jul 27 20:55:53.711074 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.710961 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e09d53c6-7715-4732-aafb-9a9b56a3f54e" containerName="extract" Jul 27 20:55:53.714145 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.714123 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw" Jul 27 20:55:53.722098 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.722061 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw"] Jul 27 20:55:53.738386 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.738361 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lfssj\"" Jul 27 20:55:53.738497 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.738386 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 27 20:55:53.738497 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.738395 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 27 20:55:53.739283 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.739267 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 27 20:55:53.840100 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.840072 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9253d899-26f8-431e-8bb3-311580627a62-cert\") pod \"lws-controller-manager-58799fcf76-s22kw\" (UID: \"9253d899-26f8-431e-8bb3-311580627a62\") " pod="openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw" Jul 27 20:55:53.840262 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.840109 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9253d899-26f8-431e-8bb3-311580627a62-manager-config\") pod \"lws-controller-manager-58799fcf76-s22kw\" (UID: \"9253d899-26f8-431e-8bb3-311580627a62\") " pod="openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw" Jul 27 20:55:53.840262 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.840141 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9253d899-26f8-431e-8bb3-311580627a62-metrics-cert\") pod \"lws-controller-manager-58799fcf76-s22kw\" (UID: \"9253d899-26f8-431e-8bb3-311580627a62\") " pod="openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw" Jul 27 20:55:53.840262 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.840191 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9vzk\" (UniqueName: \"kubernetes.io/projected/9253d899-26f8-431e-8bb3-311580627a62-kube-api-access-h9vzk\") pod \"lws-controller-manager-58799fcf76-s22kw\" (UID: \"9253d899-26f8-431e-8bb3-311580627a62\") " pod="openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw" Jul 27 20:55:53.941548 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.941515 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9253d899-26f8-431e-8bb3-311580627a62-manager-config\") pod \"lws-controller-manager-58799fcf76-s22kw\" (UID: \"9253d899-26f8-431e-8bb3-311580627a62\") " pod="openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw" Jul 27 20:55:53.941683 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.941560 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9253d899-26f8-431e-8bb3-311580627a62-metrics-cert\") pod \"lws-controller-manager-58799fcf76-s22kw\" (UID: \"9253d899-26f8-431e-8bb3-311580627a62\") " pod="openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw" Jul 27 20:55:53.941683 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.941579 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9vzk\" (UniqueName: \"kubernetes.io/projected/9253d899-26f8-431e-8bb3-311580627a62-kube-api-access-h9vzk\") pod \"lws-controller-manager-58799fcf76-s22kw\" (UID: \"9253d899-26f8-431e-8bb3-311580627a62\") " pod="openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw" Jul 27 20:55:53.941683 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.941632 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9253d899-26f8-431e-8bb3-311580627a62-cert\") pod \"lws-controller-manager-58799fcf76-s22kw\" (UID: \"9253d899-26f8-431e-8bb3-311580627a62\") " pod="openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw" Jul 27 20:55:53.942130 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.942105 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9253d899-26f8-431e-8bb3-311580627a62-manager-config\") pod \"lws-controller-manager-58799fcf76-s22kw\" (UID: \"9253d899-26f8-431e-8bb3-311580627a62\") " pod="openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw" Jul 27 20:55:53.944056 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.944033 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9253d899-26f8-431e-8bb3-311580627a62-metrics-cert\") pod \"lws-controller-manager-58799fcf76-s22kw\" (UID: \"9253d899-26f8-431e-8bb3-311580627a62\") " pod="openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw" Jul 27 20:55:53.944153 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.944108 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9253d899-26f8-431e-8bb3-311580627a62-cert\") pod \"lws-controller-manager-58799fcf76-s22kw\" (UID: \"9253d899-26f8-431e-8bb3-311580627a62\") " pod="openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw" Jul 27 20:55:53.968175 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:53.968120 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9vzk\" (UniqueName: \"kubernetes.io/projected/9253d899-26f8-431e-8bb3-311580627a62-kube-api-access-h9vzk\") pod \"lws-controller-manager-58799fcf76-s22kw\" (UID: \"9253d899-26f8-431e-8bb3-311580627a62\") " pod="openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw" Jul 27 20:55:54.024082 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:54.024043 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw" Jul 27 20:55:54.223295 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:54.223158 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw"] Jul 27 20:55:54.225558 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:55:54.225529 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9253d899_26f8_431e_8bb3_311580627a62.slice/crio-289ad8fb2391f13ff2ec11a81e0fa82251d581e8773eec27024923453de1c15a WatchSource:0}: Error finding container 289ad8fb2391f13ff2ec11a81e0fa82251d581e8773eec27024923453de1c15a: Status 404 returned error can't find the container with id 289ad8fb2391f13ff2ec11a81e0fa82251d581e8773eec27024923453de1c15a Jul 27 20:55:55.191016 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:55.190979 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw" event={"ID":"9253d899-26f8-431e-8bb3-311580627a62","Type":"ContainerStarted","Data":"289ad8fb2391f13ff2ec11a81e0fa82251d581e8773eec27024923453de1c15a"} Jul 27 20:55:56.197111 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:56.197081 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw" event={"ID":"9253d899-26f8-431e-8bb3-311580627a62","Type":"ContainerStarted","Data":"934d8197611ae16c229154c30a1b6e0c5ac9e470ec0712fb3adc3cf3479ea1d7"} Jul 27 20:55:56.197555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:56.197163 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw" Jul 27 20:55:56.257819 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:56.257748 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw" podStartSLOduration=1.8162370349999999 podStartE2EDuration="3.257727505s" totalImagesPullingTime="1.44149047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:55:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:55:56.255588382 +0000 UTC m=+284.778706127" watchObservedRunningTime="2026-07-27 20:55:56.257727505 +0000 UTC m=+284.780845230" Jul 27 20:55:57.533995 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.533963 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk"] Jul 27 20:55:57.536840 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.536823 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" Jul 27 20:55:57.545050 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.545032 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 27 20:55:57.546437 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.546421 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 27 20:55:57.547374 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.547359 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 27 20:55:57.547662 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.547649 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 27 20:55:57.547712 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.547649 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-h72pm\"" Jul 27 20:55:57.567734 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.567698 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8d30f9a9-b61d-4410-ac5c-9d41441e2ff0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk\" (UID: \"8d30f9a9-b61d-4410-ac5c-9d41441e2ff0\") " pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" Jul 27 20:55:57.567896 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.567768 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d30f9a9-b61d-4410-ac5c-9d41441e2ff0-tmp\") pod \"opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk\" (UID: \"8d30f9a9-b61d-4410-ac5c-9d41441e2ff0\") " pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" Jul 27 20:55:57.567896 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.567796 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d30f9a9-b61d-4410-ac5c-9d41441e2ff0-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk\" (UID: \"8d30f9a9-b61d-4410-ac5c-9d41441e2ff0\") " pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" Jul 27 20:55:57.567896 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.567821 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js5d8\" (UniqueName: \"kubernetes.io/projected/8d30f9a9-b61d-4410-ac5c-9d41441e2ff0-kube-api-access-js5d8\") pod \"opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk\" (UID: \"8d30f9a9-b61d-4410-ac5c-9d41441e2ff0\") " pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" Jul 27 20:55:57.567896 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.567886 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8d30f9a9-b61d-4410-ac5c-9d41441e2ff0-manifests\") pod \"opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk\" (UID: \"8d30f9a9-b61d-4410-ac5c-9d41441e2ff0\") " pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" Jul 27 20:55:57.579154 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.579124 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk"] Jul 27 20:55:57.668201 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.668162 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8d30f9a9-b61d-4410-ac5c-9d41441e2ff0-manifests\") pod \"opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk\" (UID: \"8d30f9a9-b61d-4410-ac5c-9d41441e2ff0\") " pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" Jul 27 20:55:57.668201 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.668205 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8d30f9a9-b61d-4410-ac5c-9d41441e2ff0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk\" (UID: \"8d30f9a9-b61d-4410-ac5c-9d41441e2ff0\") " pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" Jul 27 20:55:57.668430 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.668265 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d30f9a9-b61d-4410-ac5c-9d41441e2ff0-tmp\") pod \"opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk\" (UID: \"8d30f9a9-b61d-4410-ac5c-9d41441e2ff0\") " pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" Jul 27 20:55:57.668430 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.668284 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d30f9a9-b61d-4410-ac5c-9d41441e2ff0-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk\" (UID: \"8d30f9a9-b61d-4410-ac5c-9d41441e2ff0\") " pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" Jul 27 20:55:57.668430 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.668302 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js5d8\" (UniqueName: \"kubernetes.io/projected/8d30f9a9-b61d-4410-ac5c-9d41441e2ff0-kube-api-access-js5d8\") pod \"opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk\" (UID: \"8d30f9a9-b61d-4410-ac5c-9d41441e2ff0\") " pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" Jul 27 20:55:57.668738 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.668699 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8d30f9a9-b61d-4410-ac5c-9d41441e2ff0-manifests\") pod \"opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk\" (UID: \"8d30f9a9-b61d-4410-ac5c-9d41441e2ff0\") " pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" Jul 27 20:55:57.670830 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.670799 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d30f9a9-b61d-4410-ac5c-9d41441e2ff0-tmp\") pod \"opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk\" (UID: \"8d30f9a9-b61d-4410-ac5c-9d41441e2ff0\") " pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" Jul 27 20:55:57.670923 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.670865 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d30f9a9-b61d-4410-ac5c-9d41441e2ff0-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk\" (UID: \"8d30f9a9-b61d-4410-ac5c-9d41441e2ff0\") " pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" Jul 27 20:55:57.670963 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.670925 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8d30f9a9-b61d-4410-ac5c-9d41441e2ff0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk\" (UID: \"8d30f9a9-b61d-4410-ac5c-9d41441e2ff0\") " pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" Jul 27 20:55:57.768966 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.768926 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js5d8\" (UniqueName: \"kubernetes.io/projected/8d30f9a9-b61d-4410-ac5c-9d41441e2ff0-kube-api-access-js5d8\") pod \"opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk\" (UID: \"8d30f9a9-b61d-4410-ac5c-9d41441e2ff0\") " pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" Jul 27 20:55:57.847035 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:57.846946 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" Jul 27 20:55:58.037907 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:55:58.037873 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d30f9a9_b61d_4410_ac5c_9d41441e2ff0.slice/crio-5caf0036e8b65398b317d796841464ece6a044dbc79f639100600729b01da400 WatchSource:0}: Error finding container 5caf0036e8b65398b317d796841464ece6a044dbc79f639100600729b01da400: Status 404 returned error can't find the container with id 5caf0036e8b65398b317d796841464ece6a044dbc79f639100600729b01da400 Jul 27 20:55:58.045811 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:58.045788 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk"] Jul 27 20:55:58.206722 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:55:58.206691 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" event={"ID":"8d30f9a9-b61d-4410-ac5c-9d41441e2ff0","Type":"ContainerStarted","Data":"5caf0036e8b65398b317d796841464ece6a044dbc79f639100600729b01da400"} Jul 27 20:56:02.227537 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:02.227501 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" event={"ID":"8d30f9a9-b61d-4410-ac5c-9d41441e2ff0","Type":"ContainerStarted","Data":"1bbd3d9369cbccf30a65548d75e3d0d9c411d022465e11f6e0d736983d816350"} Jul 27 20:56:03.233407 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:03.233366 2565 generic.go:356] "Generic (PLEG): container finished" podID="8d30f9a9-b61d-4410-ac5c-9d41441e2ff0" containerID="1bbd3d9369cbccf30a65548d75e3d0d9c411d022465e11f6e0d736983d816350" exitCode=0 Jul 27 20:56:03.233804 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:03.233459 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" event={"ID":"8d30f9a9-b61d-4410-ac5c-9d41441e2ff0","Type":"ContainerDied","Data":"1bbd3d9369cbccf30a65548d75e3d0d9c411d022465e11f6e0d736983d816350"} Jul 27 20:56:04.239376 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:04.239342 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" event={"ID":"8d30f9a9-b61d-4410-ac5c-9d41441e2ff0","Type":"ContainerStarted","Data":"c7f9423c319e81c46f2cca98ed46537e0bd66f40772b26dcc35346ec2e3e454c"} Jul 27 20:56:04.239778 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:04.239471 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" Jul 27 20:56:04.272031 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:04.271976 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" podStartSLOduration=2.593272915 podStartE2EDuration="7.271956139s" totalImagesPullingTime="3.633063574s" totalInitContainerRuntime="1.04561965s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:55:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:56:04.270058731 +0000 UTC m=+292.793176453" watchObservedRunningTime="2026-07-27 20:56:04.271956139 +0000 UTC m=+292.795073861" Jul 27 20:56:07.204057 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:07.204028 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-58799fcf76-s22kw" Jul 27 20:56:11.912809 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:11.912782 2565 kubelet.go:1648] "Image garbage collection succeeded" Jul 27 20:56:15.246095 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:15.246064 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk" Jul 27 20:56:29.157427 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.157386 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5"] Jul 27 20:56:29.161960 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.161943 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.166754 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.166721 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-5h5pt\"" Jul 27 20:56:29.166901 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.166870 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 27 20:56:29.167091 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.167063 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 27 20:56:29.179805 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.179788 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 27 20:56:29.201757 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.201736 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.201830 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.201766 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.201830 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.201784 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.201830 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.201800 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.201830 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.201817 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.202040 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.201894 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.202040 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.201974 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.202040 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.202022 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sllxw\" (UniqueName: \"kubernetes.io/projected/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-kube-api-access-sllxw\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.202168 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.202064 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.219341 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.219318 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5"] Jul 27 20:56:29.303007 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.302980 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sllxw\" (UniqueName: \"kubernetes.io/projected/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-kube-api-access-sllxw\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.303108 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.303016 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.303108 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.303051 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.303108 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.303083 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.303272 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.303107 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.303272 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.303255 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.303374 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.303301 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.303374 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.303343 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.303476 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.303381 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.303476 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.303454 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.303585 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.303471 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.303684 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.303667 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.303751 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.303718 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.303890 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.303871 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.305507 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.305481 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.305701 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.305686 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.327989 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.327961 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.329112 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.329095 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sllxw\" (UniqueName: \"kubernetes.io/projected/f30bbd8c-d47c-442f-8fd8-87350cd7ea03-kube-api-access-sllxw\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5\" (UID: \"f30bbd8c-d47c-442f-8fd8-87350cd7ea03\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.472525 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.472490 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:29.645378 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.645354 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5"] Jul 27 20:56:29.647462 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:56:29.647437 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf30bbd8c_d47c_442f_8fd8_87350cd7ea03.slice/crio-90a39b10103cdfdb016cce2277805f27ee0bce45eca4c67d846562e761bba183 WatchSource:0}: Error finding container 90a39b10103cdfdb016cce2277805f27ee0bce45eca4c67d846562e761bba183: Status 404 returned error can't find the container with id 90a39b10103cdfdb016cce2277805f27ee0bce45eca4c67d846562e761bba183 Jul 27 20:56:29.649320 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:29.649304 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 20:56:30.353830 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:30.353794 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" event={"ID":"f30bbd8c-d47c-442f-8fd8-87350cd7ea03","Type":"ContainerStarted","Data":"90a39b10103cdfdb016cce2277805f27ee0bce45eca4c67d846562e761bba183"} Jul 27 20:56:33.069214 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:33.069170 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 20:56:33.069517 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:33.069269 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 20:56:33.069517 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:33.069297 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 20:56:33.371172 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:33.371092 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" event={"ID":"f30bbd8c-d47c-442f-8fd8-87350cd7ea03","Type":"ContainerStarted","Data":"e58254e6f08d5bf9586b4e2f9194396bd90840549e8c3a0beadcfab6be0b2a07"} Jul 27 20:56:33.434467 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:33.434409 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" podStartSLOduration=1.014904192 podStartE2EDuration="4.434393375s" totalImagesPullingTime="3.419489183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:56:33.432154836 +0000 UTC m=+321.955272558" watchObservedRunningTime="2026-07-27 20:56:33.434393375 +0000 UTC m=+321.957511097" Jul 27 20:56:33.473121 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:33.473093 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:34.478545 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:34.478517 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:35.379417 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:35.379388 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:35.380398 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:35.380380 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5" Jul 27 20:56:35.853175 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:35.853145 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-72bqn"] Jul 27 20:56:35.860744 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:35.860715 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72bqn" Jul 27 20:56:35.871605 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:35.871579 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 27 20:56:35.871715 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:35.871610 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-r7bcq\"" Jul 27 20:56:35.960716 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:35.960683 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfw2l\" (UniqueName: \"kubernetes.io/projected/b5268780-a4cb-41e2-9775-5a50c9d84f24-kube-api-access-nfw2l\") pod \"llmisvc-controller-manager-777c6bd7d5-72bqn\" (UID: \"b5268780-a4cb-41e2-9775-5a50c9d84f24\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72bqn" Jul 27 20:56:35.960856 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:35.960729 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5268780-a4cb-41e2-9775-5a50c9d84f24-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-72bqn\" (UID: \"b5268780-a4cb-41e2-9775-5a50c9d84f24\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72bqn" Jul 27 20:56:36.010374 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:36.010344 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-72bqn"] Jul 27 20:56:36.061567 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:36.061534 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfw2l\" (UniqueName: \"kubernetes.io/projected/b5268780-a4cb-41e2-9775-5a50c9d84f24-kube-api-access-nfw2l\") pod \"llmisvc-controller-manager-777c6bd7d5-72bqn\" (UID: \"b5268780-a4cb-41e2-9775-5a50c9d84f24\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72bqn" Jul 27 20:56:36.061714 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:36.061597 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5268780-a4cb-41e2-9775-5a50c9d84f24-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-72bqn\" (UID: \"b5268780-a4cb-41e2-9775-5a50c9d84f24\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72bqn" Jul 27 20:56:36.064003 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:36.063979 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5268780-a4cb-41e2-9775-5a50c9d84f24-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-72bqn\" (UID: \"b5268780-a4cb-41e2-9775-5a50c9d84f24\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72bqn" Jul 27 20:56:36.087964 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:36.087933 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfw2l\" (UniqueName: \"kubernetes.io/projected/b5268780-a4cb-41e2-9775-5a50c9d84f24-kube-api-access-nfw2l\") pod \"llmisvc-controller-manager-777c6bd7d5-72bqn\" (UID: \"b5268780-a4cb-41e2-9775-5a50c9d84f24\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72bqn" Jul 27 20:56:36.170308 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:36.170282 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72bqn" Jul 27 20:56:36.424436 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:56:36.424407 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb5268780_a4cb_41e2_9775_5a50c9d84f24.slice/crio-5f87c17fe076237df9666a7b7424276f61ec7fb158121679b15ab72abd678ee7 WatchSource:0}: Error finding container 5f87c17fe076237df9666a7b7424276f61ec7fb158121679b15ab72abd678ee7: Status 404 returned error can't find the container with id 5f87c17fe076237df9666a7b7424276f61ec7fb158121679b15ab72abd678ee7 Jul 27 20:56:36.480022 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:36.479999 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-72bqn"] Jul 27 20:56:37.389268 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:37.389206 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72bqn" event={"ID":"b5268780-a4cb-41e2-9775-5a50c9d84f24","Type":"ContainerStarted","Data":"5f87c17fe076237df9666a7b7424276f61ec7fb158121679b15ab72abd678ee7"} Jul 27 20:56:39.398548 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:39.398517 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72bqn" event={"ID":"b5268780-a4cb-41e2-9775-5a50c9d84f24","Type":"ContainerStarted","Data":"1a952dce1fc086617b00e86be4c7057ac81dc23c8e329d35d5982fc14c4132e2"} Jul 27 20:56:39.398916 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:56:39.398637 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72bqn" Jul 27 20:57:10.404474 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:10.404390 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72bqn" Jul 27 20:57:10.435600 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:10.435553 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72bqn" podStartSLOduration=32.657256541 podStartE2EDuration="35.435537954s" totalImagesPullingTime="2.778281413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:56:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:56:39.436875802 +0000 UTC m=+327.959993522" watchObservedRunningTime="2026-07-27 20:57:10.435537954 +0000 UTC m=+358.958655676" Jul 27 20:57:19.355549 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:19.355510 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j98kz"] Jul 27 20:57:19.367807 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:19.367777 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j98kz" Jul 27 20:57:19.369121 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:19.369094 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j98kz"] Jul 27 20:57:19.371312 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:19.371280 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 27 20:57:19.371532 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:19.371505 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 27 20:57:19.372728 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:19.372702 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-t6tqz\"" Jul 27 20:57:19.411052 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:19.411019 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l2mf\" (UniqueName: \"kubernetes.io/projected/f0163b79-ff0b-4f14-badf-4af540841cc3-kube-api-access-7l2mf\") pod \"kuadrant-operator-catalog-j98kz\" (UID: \"f0163b79-ff0b-4f14-badf-4af540841cc3\") " pod="kuadrant-system/kuadrant-operator-catalog-j98kz" Jul 27 20:57:19.511921 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:19.511880 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l2mf\" (UniqueName: \"kubernetes.io/projected/f0163b79-ff0b-4f14-badf-4af540841cc3-kube-api-access-7l2mf\") pod \"kuadrant-operator-catalog-j98kz\" (UID: \"f0163b79-ff0b-4f14-badf-4af540841cc3\") " pod="kuadrant-system/kuadrant-operator-catalog-j98kz" Jul 27 20:57:19.520416 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:19.520395 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l2mf\" (UniqueName: \"kubernetes.io/projected/f0163b79-ff0b-4f14-badf-4af540841cc3-kube-api-access-7l2mf\") pod \"kuadrant-operator-catalog-j98kz\" (UID: \"f0163b79-ff0b-4f14-badf-4af540841cc3\") " pod="kuadrant-system/kuadrant-operator-catalog-j98kz" Jul 27 20:57:19.678827 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:19.678796 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j98kz" Jul 27 20:57:19.721080 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:19.721047 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j98kz"] Jul 27 20:57:19.806513 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:19.806489 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j98kz"] Jul 27 20:57:19.808335 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:57:19.808303 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0163b79_ff0b_4f14_badf_4af540841cc3.slice/crio-a7aae0391dd6bcb5060c6c5faa5194d3937a4c0ee437dc11601aa31c200c3cc5 WatchSource:0}: Error finding container a7aae0391dd6bcb5060c6c5faa5194d3937a4c0ee437dc11601aa31c200c3cc5: Status 404 returned error can't find the container with id a7aae0391dd6bcb5060c6c5faa5194d3937a4c0ee437dc11601aa31c200c3cc5 Jul 27 20:57:19.926682 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:19.926654 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6xgc4"] Jul 27 20:57:19.931274 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:19.931218 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6xgc4" Jul 27 20:57:19.937735 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:19.937712 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6xgc4"] Jul 27 20:57:20.016400 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:20.016368 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlx6j\" (UniqueName: \"kubernetes.io/projected/07577e4f-3f3e-4399-b803-823cc453a561-kube-api-access-qlx6j\") pod \"kuadrant-operator-catalog-6xgc4\" (UID: \"07577e4f-3f3e-4399-b803-823cc453a561\") " pod="kuadrant-system/kuadrant-operator-catalog-6xgc4" Jul 27 20:57:20.116985 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:20.116947 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlx6j\" (UniqueName: \"kubernetes.io/projected/07577e4f-3f3e-4399-b803-823cc453a561-kube-api-access-qlx6j\") pod \"kuadrant-operator-catalog-6xgc4\" (UID: \"07577e4f-3f3e-4399-b803-823cc453a561\") " pod="kuadrant-system/kuadrant-operator-catalog-6xgc4" Jul 27 20:57:20.132776 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:20.132744 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlx6j\" (UniqueName: \"kubernetes.io/projected/07577e4f-3f3e-4399-b803-823cc453a561-kube-api-access-qlx6j\") pod \"kuadrant-operator-catalog-6xgc4\" (UID: \"07577e4f-3f3e-4399-b803-823cc453a561\") " pod="kuadrant-system/kuadrant-operator-catalog-6xgc4" Jul 27 20:57:20.241107 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:20.241012 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6xgc4" Jul 27 20:57:20.387227 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:20.387202 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6xgc4"] Jul 27 20:57:20.388567 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:57:20.388539 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07577e4f_3f3e_4399_b803_823cc453a561.slice/crio-51a8e18feee4b6e6f1bcf1aeccbb72008730bea4faf6c770a7f88412aaa838ff WatchSource:0}: Error finding container 51a8e18feee4b6e6f1bcf1aeccbb72008730bea4faf6c770a7f88412aaa838ff: Status 404 returned error can't find the container with id 51a8e18feee4b6e6f1bcf1aeccbb72008730bea4faf6c770a7f88412aaa838ff Jul 27 20:57:20.580634 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:20.580566 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j98kz" event={"ID":"f0163b79-ff0b-4f14-badf-4af540841cc3","Type":"ContainerStarted","Data":"a7aae0391dd6bcb5060c6c5faa5194d3937a4c0ee437dc11601aa31c200c3cc5"} Jul 27 20:57:20.581712 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:20.581687 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6xgc4" event={"ID":"07577e4f-3f3e-4399-b803-823cc453a561","Type":"ContainerStarted","Data":"51a8e18feee4b6e6f1bcf1aeccbb72008730bea4faf6c770a7f88412aaa838ff"} Jul 27 20:57:22.592716 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:22.592678 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j98kz" event={"ID":"f0163b79-ff0b-4f14-badf-4af540841cc3","Type":"ContainerStarted","Data":"8f4c179ac3629c20c8ad36f9976669d754b47c91e24241395c4f79538019b147"} Jul 27 20:57:22.593181 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:22.592740 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-j98kz" podUID="f0163b79-ff0b-4f14-badf-4af540841cc3" containerName="registry-server" containerID="cri-o://8f4c179ac3629c20c8ad36f9976669d754b47c91e24241395c4f79538019b147" gracePeriod=2 Jul 27 20:57:22.594368 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:22.594342 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6xgc4" event={"ID":"07577e4f-3f3e-4399-b803-823cc453a561","Type":"ContainerStarted","Data":"3e6c383b2030be834ca4177609926e9139e01715a35febf7359bd07b8e6481bf"} Jul 27 20:57:22.613334 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:22.613290 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-j98kz" podStartSLOduration=1.515950337 podStartE2EDuration="3.613274791s" totalImagesPullingTime="2.097324454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:57:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:57:22.612052873 +0000 UTC m=+371.135170597" watchObservedRunningTime="2026-07-27 20:57:22.613274791 +0000 UTC m=+371.136392512" Jul 27 20:57:22.635784 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:22.635739 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-6xgc4" podStartSLOduration=2.08230816 podStartE2EDuration="3.635724635s" totalImagesPullingTime="1.553416475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:57:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:57:22.63547433 +0000 UTC m=+371.158592054" watchObservedRunningTime="2026-07-27 20:57:22.635724635 +0000 UTC m=+371.158842357" Jul 27 20:57:22.846027 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:22.845975 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j98kz" Jul 27 20:57:22.940691 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:22.940647 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7l2mf\" (UniqueName: \"kubernetes.io/projected/f0163b79-ff0b-4f14-badf-4af540841cc3-kube-api-access-7l2mf\") pod \"f0163b79-ff0b-4f14-badf-4af540841cc3\" (UID: \"f0163b79-ff0b-4f14-badf-4af540841cc3\") " Jul 27 20:57:22.942949 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:22.942916 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0163b79-ff0b-4f14-badf-4af540841cc3-kube-api-access-7l2mf" (OuterVolumeSpecName: "kube-api-access-7l2mf") pod "f0163b79-ff0b-4f14-badf-4af540841cc3" (UID: "f0163b79-ff0b-4f14-badf-4af540841cc3"). InnerVolumeSpecName "kube-api-access-7l2mf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 20:57:23.042126 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:23.042092 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7l2mf\" (UniqueName: \"kubernetes.io/projected/f0163b79-ff0b-4f14-badf-4af540841cc3-kube-api-access-7l2mf\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:57:23.601009 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:23.600967 2565 generic.go:356] "Generic (PLEG): container finished" podID="f0163b79-ff0b-4f14-badf-4af540841cc3" containerID="8f4c179ac3629c20c8ad36f9976669d754b47c91e24241395c4f79538019b147" exitCode=0 Jul 27 20:57:23.601009 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:23.601011 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j98kz" Jul 27 20:57:23.601547 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:23.601015 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j98kz" event={"ID":"f0163b79-ff0b-4f14-badf-4af540841cc3","Type":"ContainerDied","Data":"8f4c179ac3629c20c8ad36f9976669d754b47c91e24241395c4f79538019b147"} Jul 27 20:57:23.601547 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:23.601091 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j98kz" event={"ID":"f0163b79-ff0b-4f14-badf-4af540841cc3","Type":"ContainerDied","Data":"a7aae0391dd6bcb5060c6c5faa5194d3937a4c0ee437dc11601aa31c200c3cc5"} Jul 27 20:57:23.601547 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:23.601121 2565 scope.go:117] "RemoveContainer" containerID="8f4c179ac3629c20c8ad36f9976669d754b47c91e24241395c4f79538019b147" Jul 27 20:57:23.610321 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:23.610303 2565 scope.go:117] "RemoveContainer" containerID="8f4c179ac3629c20c8ad36f9976669d754b47c91e24241395c4f79538019b147" Jul 27 20:57:23.610579 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:57:23.610562 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f4c179ac3629c20c8ad36f9976669d754b47c91e24241395c4f79538019b147\": container with ID starting with 8f4c179ac3629c20c8ad36f9976669d754b47c91e24241395c4f79538019b147 not found: ID does not exist" containerID="8f4c179ac3629c20c8ad36f9976669d754b47c91e24241395c4f79538019b147" Jul 27 20:57:23.610627 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:23.610588 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f4c179ac3629c20c8ad36f9976669d754b47c91e24241395c4f79538019b147"} err="failed to get container status \"8f4c179ac3629c20c8ad36f9976669d754b47c91e24241395c4f79538019b147\": rpc error: code = NotFound desc = could not find container \"8f4c179ac3629c20c8ad36f9976669d754b47c91e24241395c4f79538019b147\": container with ID starting with 8f4c179ac3629c20c8ad36f9976669d754b47c91e24241395c4f79538019b147 not found: ID does not exist" Jul 27 20:57:23.622769 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:23.622748 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j98kz"] Jul 27 20:57:23.625149 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:23.625129 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j98kz"] Jul 27 20:57:24.052556 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:24.052521 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0163b79-ff0b-4f14-badf-4af540841cc3" path="/var/lib/kubelet/pods/f0163b79-ff0b-4f14-badf-4af540841cc3/volumes" Jul 27 20:57:30.241768 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:30.241736 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-6xgc4" Jul 27 20:57:30.242154 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:30.241795 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-6xgc4" Jul 27 20:57:30.264189 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:30.264161 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-6xgc4" Jul 27 20:57:30.650564 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:30.650495 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-6xgc4" Jul 27 20:57:35.754868 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:35.754833 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th"] Jul 27 20:57:35.755287 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:35.755131 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0163b79-ff0b-4f14-badf-4af540841cc3" containerName="registry-server" Jul 27 20:57:35.755287 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:35.755143 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0163b79-ff0b-4f14-badf-4af540841cc3" containerName="registry-server" Jul 27 20:57:35.755287 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:35.755204 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0163b79-ff0b-4f14-badf-4af540841cc3" containerName="registry-server" Jul 27 20:57:35.758041 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:35.758025 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th" Jul 27 20:57:35.760661 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:35.760638 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-t6w6q\"" Jul 27 20:57:35.766669 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:35.766648 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th"] Jul 27 20:57:35.844482 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:35.844441 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c027e754-ca8e-4a2b-8ca1-60677abd3f1b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th\" (UID: \"c027e754-ca8e-4a2b-8ca1-60677abd3f1b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th" Jul 27 20:57:35.844482 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:35.844483 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c027e754-ca8e-4a2b-8ca1-60677abd3f1b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th\" (UID: \"c027e754-ca8e-4a2b-8ca1-60677abd3f1b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th" Jul 27 20:57:35.844696 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:35.844552 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl74n\" (UniqueName: \"kubernetes.io/projected/c027e754-ca8e-4a2b-8ca1-60677abd3f1b-kube-api-access-dl74n\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th\" (UID: \"c027e754-ca8e-4a2b-8ca1-60677abd3f1b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th" Jul 27 20:57:35.945436 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:35.945403 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl74n\" (UniqueName: \"kubernetes.io/projected/c027e754-ca8e-4a2b-8ca1-60677abd3f1b-kube-api-access-dl74n\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th\" (UID: \"c027e754-ca8e-4a2b-8ca1-60677abd3f1b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th" Jul 27 20:57:35.945555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:35.945459 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c027e754-ca8e-4a2b-8ca1-60677abd3f1b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th\" (UID: \"c027e754-ca8e-4a2b-8ca1-60677abd3f1b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th" Jul 27 20:57:35.945555 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:35.945479 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c027e754-ca8e-4a2b-8ca1-60677abd3f1b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th\" (UID: \"c027e754-ca8e-4a2b-8ca1-60677abd3f1b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th" Jul 27 20:57:35.945784 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:35.945770 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c027e754-ca8e-4a2b-8ca1-60677abd3f1b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th\" (UID: \"c027e754-ca8e-4a2b-8ca1-60677abd3f1b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th" Jul 27 20:57:35.945820 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:35.945797 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c027e754-ca8e-4a2b-8ca1-60677abd3f1b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th\" (UID: \"c027e754-ca8e-4a2b-8ca1-60677abd3f1b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th" Jul 27 20:57:35.954320 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:35.954295 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl74n\" (UniqueName: \"kubernetes.io/projected/c027e754-ca8e-4a2b-8ca1-60677abd3f1b-kube-api-access-dl74n\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th\" (UID: \"c027e754-ca8e-4a2b-8ca1-60677abd3f1b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th" Jul 27 20:57:36.067469 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.067394 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th" Jul 27 20:57:36.160410 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.160380 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf"] Jul 27 20:57:36.164587 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.164566 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf" Jul 27 20:57:36.173724 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.173702 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf"] Jul 27 20:57:36.200728 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.200707 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th"] Jul 27 20:57:36.203033 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:57:36.203007 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc027e754_ca8e_4a2b_8ca1_60677abd3f1b.slice/crio-e9bf15092152c23e300c4210605b65f8d45067fddf97fc8bc0dec680a38494dc WatchSource:0}: Error finding container e9bf15092152c23e300c4210605b65f8d45067fddf97fc8bc0dec680a38494dc: Status 404 returned error can't find the container with id e9bf15092152c23e300c4210605b65f8d45067fddf97fc8bc0dec680a38494dc Jul 27 20:57:36.247839 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.247815 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b45875d-c575-4fb8-985d-ae556d78565e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf\" (UID: \"8b45875d-c575-4fb8-985d-ae556d78565e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf" Jul 27 20:57:36.247940 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.247886 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlsm9\" (UniqueName: \"kubernetes.io/projected/8b45875d-c575-4fb8-985d-ae556d78565e-kube-api-access-zlsm9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf\" (UID: \"8b45875d-c575-4fb8-985d-ae556d78565e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf" Jul 27 20:57:36.248006 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.247981 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b45875d-c575-4fb8-985d-ae556d78565e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf\" (UID: \"8b45875d-c575-4fb8-985d-ae556d78565e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf" Jul 27 20:57:36.349269 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.349184 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlsm9\" (UniqueName: \"kubernetes.io/projected/8b45875d-c575-4fb8-985d-ae556d78565e-kube-api-access-zlsm9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf\" (UID: \"8b45875d-c575-4fb8-985d-ae556d78565e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf" Jul 27 20:57:36.349269 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.349225 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b45875d-c575-4fb8-985d-ae556d78565e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf\" (UID: \"8b45875d-c575-4fb8-985d-ae556d78565e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf" Jul 27 20:57:36.349452 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.349363 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b45875d-c575-4fb8-985d-ae556d78565e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf\" (UID: \"8b45875d-c575-4fb8-985d-ae556d78565e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf" Jul 27 20:57:36.349530 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.349516 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b45875d-c575-4fb8-985d-ae556d78565e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf\" (UID: \"8b45875d-c575-4fb8-985d-ae556d78565e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf" Jul 27 20:57:36.349667 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.349650 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b45875d-c575-4fb8-985d-ae556d78565e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf\" (UID: \"8b45875d-c575-4fb8-985d-ae556d78565e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf" Jul 27 20:57:36.357453 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.357435 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlsm9\" (UniqueName: \"kubernetes.io/projected/8b45875d-c575-4fb8-985d-ae556d78565e-kube-api-access-zlsm9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf\" (UID: \"8b45875d-c575-4fb8-985d-ae556d78565e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf" Jul 27 20:57:36.475904 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.475867 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf" Jul 27 20:57:36.604195 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.604167 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf"] Jul 27 20:57:36.605627 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:57:36.605598 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b45875d_c575_4fb8_985d_ae556d78565e.slice/crio-247a5d8977c1c09861bb584a9c89c7f0a3f118e9ef80e7e906350adc7ae9b89e WatchSource:0}: Error finding container 247a5d8977c1c09861bb584a9c89c7f0a3f118e9ef80e7e906350adc7ae9b89e: Status 404 returned error can't find the container with id 247a5d8977c1c09861bb584a9c89c7f0a3f118e9ef80e7e906350adc7ae9b89e Jul 27 20:57:36.657994 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.657959 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf" event={"ID":"8b45875d-c575-4fb8-985d-ae556d78565e","Type":"ContainerStarted","Data":"247a5d8977c1c09861bb584a9c89c7f0a3f118e9ef80e7e906350adc7ae9b89e"} Jul 27 20:57:36.659357 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.659335 2565 generic.go:356] "Generic (PLEG): container finished" podID="c027e754-ca8e-4a2b-8ca1-60677abd3f1b" containerID="9c7986ea0919b1f1c1a43bd6b19027fcbc8cc0f5b461727de1898bfc500026a4" exitCode=0 Jul 27 20:57:36.659442 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.659421 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th" event={"ID":"c027e754-ca8e-4a2b-8ca1-60677abd3f1b","Type":"ContainerDied","Data":"9c7986ea0919b1f1c1a43bd6b19027fcbc8cc0f5b461727de1898bfc500026a4"} Jul 27 20:57:36.659491 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:36.659463 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th" event={"ID":"c027e754-ca8e-4a2b-8ca1-60677abd3f1b","Type":"ContainerStarted","Data":"e9bf15092152c23e300c4210605b65f8d45067fddf97fc8bc0dec680a38494dc"} Jul 27 20:57:37.665360 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:37.665329 2565 generic.go:356] "Generic (PLEG): container finished" podID="8b45875d-c575-4fb8-985d-ae556d78565e" containerID="d89eeb2c779e366aceac1a5066e25bfbeb3a6a7a054e7d8ca770d2c32bfa01c3" exitCode=0 Jul 27 20:57:37.665969 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:37.665424 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf" event={"ID":"8b45875d-c575-4fb8-985d-ae556d78565e","Type":"ContainerDied","Data":"d89eeb2c779e366aceac1a5066e25bfbeb3a6a7a054e7d8ca770d2c32bfa01c3"} Jul 27 20:57:37.667746 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:37.667721 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th" event={"ID":"c027e754-ca8e-4a2b-8ca1-60677abd3f1b","Type":"ContainerStarted","Data":"6b0b3addf16aacd528b17aba4fbdb4ee8f257f5e0c6f1c258d1c7950b77bdd82"} Jul 27 20:57:38.673473 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:38.673438 2565 generic.go:356] "Generic (PLEG): container finished" podID="c027e754-ca8e-4a2b-8ca1-60677abd3f1b" containerID="6b0b3addf16aacd528b17aba4fbdb4ee8f257f5e0c6f1c258d1c7950b77bdd82" exitCode=0 Jul 27 20:57:38.673904 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:38.673533 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th" event={"ID":"c027e754-ca8e-4a2b-8ca1-60677abd3f1b","Type":"ContainerDied","Data":"6b0b3addf16aacd528b17aba4fbdb4ee8f257f5e0c6f1c258d1c7950b77bdd82"} Jul 27 20:57:38.675539 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:38.675517 2565 generic.go:356] "Generic (PLEG): container finished" podID="8b45875d-c575-4fb8-985d-ae556d78565e" containerID="24a977a110433a59146b7f7e64607fc82c1ef9a185bc290f2cd9ec483413fb12" exitCode=0 Jul 27 20:57:38.675621 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:38.675554 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf" event={"ID":"8b45875d-c575-4fb8-985d-ae556d78565e","Type":"ContainerDied","Data":"24a977a110433a59146b7f7e64607fc82c1ef9a185bc290f2cd9ec483413fb12"} Jul 27 20:57:39.687311 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:39.687280 2565 generic.go:356] "Generic (PLEG): container finished" podID="8b45875d-c575-4fb8-985d-ae556d78565e" containerID="bc8d3694645aeeb1cc29794f3be40f60892b12283ad82a6912a95409ae7dabc4" exitCode=0 Jul 27 20:57:39.687767 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:39.687368 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf" event={"ID":"8b45875d-c575-4fb8-985d-ae556d78565e","Type":"ContainerDied","Data":"bc8d3694645aeeb1cc29794f3be40f60892b12283ad82a6912a95409ae7dabc4"} Jul 27 20:57:39.689316 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:39.689292 2565 generic.go:356] "Generic (PLEG): container finished" podID="c027e754-ca8e-4a2b-8ca1-60677abd3f1b" containerID="a2d57a03708b7ed6cde80869aa2342317adb5195e5abe253381d21b1680430c9" exitCode=0 Jul 27 20:57:39.689419 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:39.689374 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th" event={"ID":"c027e754-ca8e-4a2b-8ca1-60677abd3f1b","Type":"ContainerDied","Data":"a2d57a03708b7ed6cde80869aa2342317adb5195e5abe253381d21b1680430c9"} Jul 27 20:57:40.829673 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:40.829643 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th" Jul 27 20:57:40.851867 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:40.851835 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf" Jul 27 20:57:40.890306 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:40.890279 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dl74n\" (UniqueName: \"kubernetes.io/projected/c027e754-ca8e-4a2b-8ca1-60677abd3f1b-kube-api-access-dl74n\") pod \"c027e754-ca8e-4a2b-8ca1-60677abd3f1b\" (UID: \"c027e754-ca8e-4a2b-8ca1-60677abd3f1b\") " Jul 27 20:57:40.890466 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:40.890316 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c027e754-ca8e-4a2b-8ca1-60677abd3f1b-util\") pod \"c027e754-ca8e-4a2b-8ca1-60677abd3f1b\" (UID: \"c027e754-ca8e-4a2b-8ca1-60677abd3f1b\") " Jul 27 20:57:40.890466 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:40.890345 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c027e754-ca8e-4a2b-8ca1-60677abd3f1b-bundle\") pod \"c027e754-ca8e-4a2b-8ca1-60677abd3f1b\" (UID: \"c027e754-ca8e-4a2b-8ca1-60677abd3f1b\") " Jul 27 20:57:40.890850 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:40.890818 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c027e754-ca8e-4a2b-8ca1-60677abd3f1b-bundle" (OuterVolumeSpecName: "bundle") pod "c027e754-ca8e-4a2b-8ca1-60677abd3f1b" (UID: "c027e754-ca8e-4a2b-8ca1-60677abd3f1b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 20:57:40.892613 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:40.892586 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c027e754-ca8e-4a2b-8ca1-60677abd3f1b-kube-api-access-dl74n" (OuterVolumeSpecName: "kube-api-access-dl74n") pod "c027e754-ca8e-4a2b-8ca1-60677abd3f1b" (UID: "c027e754-ca8e-4a2b-8ca1-60677abd3f1b"). InnerVolumeSpecName "kube-api-access-dl74n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 20:57:40.896411 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:40.896384 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c027e754-ca8e-4a2b-8ca1-60677abd3f1b-util" (OuterVolumeSpecName: "util") pod "c027e754-ca8e-4a2b-8ca1-60677abd3f1b" (UID: "c027e754-ca8e-4a2b-8ca1-60677abd3f1b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 20:57:40.991273 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:40.991159 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b45875d-c575-4fb8-985d-ae556d78565e-bundle\") pod \"8b45875d-c575-4fb8-985d-ae556d78565e\" (UID: \"8b45875d-c575-4fb8-985d-ae556d78565e\") " Jul 27 20:57:40.991273 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:40.991259 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlsm9\" (UniqueName: \"kubernetes.io/projected/8b45875d-c575-4fb8-985d-ae556d78565e-kube-api-access-zlsm9\") pod \"8b45875d-c575-4fb8-985d-ae556d78565e\" (UID: \"8b45875d-c575-4fb8-985d-ae556d78565e\") " Jul 27 20:57:40.991453 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:40.991287 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b45875d-c575-4fb8-985d-ae556d78565e-util\") pod \"8b45875d-c575-4fb8-985d-ae556d78565e\" (UID: \"8b45875d-c575-4fb8-985d-ae556d78565e\") " Jul 27 20:57:40.991532 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:40.991516 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dl74n\" (UniqueName: \"kubernetes.io/projected/c027e754-ca8e-4a2b-8ca1-60677abd3f1b-kube-api-access-dl74n\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:57:40.991572 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:40.991540 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c027e754-ca8e-4a2b-8ca1-60677abd3f1b-util\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:57:40.991572 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:40.991556 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c027e754-ca8e-4a2b-8ca1-60677abd3f1b-bundle\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:57:40.991821 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:40.991803 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b45875d-c575-4fb8-985d-ae556d78565e-bundle" (OuterVolumeSpecName: "bundle") pod "8b45875d-c575-4fb8-985d-ae556d78565e" (UID: "8b45875d-c575-4fb8-985d-ae556d78565e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 20:57:40.993462 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:40.993435 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b45875d-c575-4fb8-985d-ae556d78565e-kube-api-access-zlsm9" (OuterVolumeSpecName: "kube-api-access-zlsm9") pod "8b45875d-c575-4fb8-985d-ae556d78565e" (UID: "8b45875d-c575-4fb8-985d-ae556d78565e"). InnerVolumeSpecName "kube-api-access-zlsm9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 20:57:40.996700 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:40.996658 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b45875d-c575-4fb8-985d-ae556d78565e-util" (OuterVolumeSpecName: "util") pod "8b45875d-c575-4fb8-985d-ae556d78565e" (UID: "8b45875d-c575-4fb8-985d-ae556d78565e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 20:57:41.092828 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:41.092783 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b45875d-c575-4fb8-985d-ae556d78565e-bundle\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:57:41.092828 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:41.092818 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlsm9\" (UniqueName: \"kubernetes.io/projected/8b45875d-c575-4fb8-985d-ae556d78565e-kube-api-access-zlsm9\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:57:41.092828 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:41.092828 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b45875d-c575-4fb8-985d-ae556d78565e-util\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:57:41.701054 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:41.701020 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf" Jul 27 20:57:41.701338 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:41.701021 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf" event={"ID":"8b45875d-c575-4fb8-985d-ae556d78565e","Type":"ContainerDied","Data":"247a5d8977c1c09861bb584a9c89c7f0a3f118e9ef80e7e906350adc7ae9b89e"} Jul 27 20:57:41.701338 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:41.701153 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="247a5d8977c1c09861bb584a9c89c7f0a3f118e9ef80e7e906350adc7ae9b89e" Jul 27 20:57:41.702965 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:41.702943 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th" Jul 27 20:57:41.703070 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:41.702941 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th" event={"ID":"c027e754-ca8e-4a2b-8ca1-60677abd3f1b","Type":"ContainerDied","Data":"e9bf15092152c23e300c4210605b65f8d45067fddf97fc8bc0dec680a38494dc"} Jul 27 20:57:41.703070 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:41.703056 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9bf15092152c23e300c4210605b65f8d45067fddf97fc8bc0dec680a38494dc" Jul 27 20:57:55.694654 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.694620 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-khcmr"] Jul 27 20:57:55.695010 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.694928 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c027e754-ca8e-4a2b-8ca1-60677abd3f1b" containerName="extract" Jul 27 20:57:55.695010 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.694939 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c027e754-ca8e-4a2b-8ca1-60677abd3f1b" containerName="extract" Jul 27 20:57:55.695010 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.694947 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c027e754-ca8e-4a2b-8ca1-60677abd3f1b" containerName="pull" Jul 27 20:57:55.695010 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.694952 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c027e754-ca8e-4a2b-8ca1-60677abd3f1b" containerName="pull" Jul 27 20:57:55.695010 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.694964 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b45875d-c575-4fb8-985d-ae556d78565e" containerName="pull" Jul 27 20:57:55.695010 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.694970 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b45875d-c575-4fb8-985d-ae556d78565e" containerName="pull" Jul 27 20:57:55.695010 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.694977 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b45875d-c575-4fb8-985d-ae556d78565e" containerName="extract" Jul 27 20:57:55.695010 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.694982 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b45875d-c575-4fb8-985d-ae556d78565e" containerName="extract" Jul 27 20:57:55.695010 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.694991 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c027e754-ca8e-4a2b-8ca1-60677abd3f1b" containerName="util" Jul 27 20:57:55.695010 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.694996 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c027e754-ca8e-4a2b-8ca1-60677abd3f1b" containerName="util" Jul 27 20:57:55.695010 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.695004 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b45875d-c575-4fb8-985d-ae556d78565e" containerName="util" Jul 27 20:57:55.695010 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.695008 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b45875d-c575-4fb8-985d-ae556d78565e" containerName="util" Jul 27 20:57:55.695414 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.695054 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c027e754-ca8e-4a2b-8ca1-60677abd3f1b" containerName="extract" Jul 27 20:57:55.695414 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.695064 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b45875d-c575-4fb8-985d-ae556d78565e" containerName="extract" Jul 27 20:57:55.701395 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.701378 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-khcmr" Jul 27 20:57:55.706529 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.706507 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-ttmcj\"" Jul 27 20:57:55.711995 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.711972 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-khcmr"] Jul 27 20:57:55.819112 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.819067 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48n57\" (UniqueName: \"kubernetes.io/projected/580c2387-4c06-46cc-bb8c-38ddb73724bc-kube-api-access-48n57\") pod \"authorino-operator-75cc8d5cb7-khcmr\" (UID: \"580c2387-4c06-46cc-bb8c-38ddb73724bc\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-khcmr" Jul 27 20:57:55.920272 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.920217 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48n57\" (UniqueName: \"kubernetes.io/projected/580c2387-4c06-46cc-bb8c-38ddb73724bc-kube-api-access-48n57\") pod \"authorino-operator-75cc8d5cb7-khcmr\" (UID: \"580c2387-4c06-46cc-bb8c-38ddb73724bc\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-khcmr" Jul 27 20:57:55.936531 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:55.936498 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48n57\" (UniqueName: \"kubernetes.io/projected/580c2387-4c06-46cc-bb8c-38ddb73724bc-kube-api-access-48n57\") pod \"authorino-operator-75cc8d5cb7-khcmr\" (UID: \"580c2387-4c06-46cc-bb8c-38ddb73724bc\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-khcmr" Jul 27 20:57:56.011891 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:56.011803 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-khcmr" Jul 27 20:57:56.174615 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:56.174590 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-khcmr"] Jul 27 20:57:56.176135 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:57:56.176107 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod580c2387_4c06_46cc_bb8c_38ddb73724bc.slice/crio-8116f82bd247a7e6a6c2be2d3fad996e2782ea5492fdd17949c8b341d83a7e39 WatchSource:0}: Error finding container 8116f82bd247a7e6a6c2be2d3fad996e2782ea5492fdd17949c8b341d83a7e39: Status 404 returned error can't find the container with id 8116f82bd247a7e6a6c2be2d3fad996e2782ea5492fdd17949c8b341d83a7e39 Jul 27 20:57:56.772257 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:56.772202 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-khcmr" event={"ID":"580c2387-4c06-46cc-bb8c-38ddb73724bc","Type":"ContainerStarted","Data":"8116f82bd247a7e6a6c2be2d3fad996e2782ea5492fdd17949c8b341d83a7e39"} Jul 27 20:57:58.784456 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:58.784419 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-khcmr" event={"ID":"580c2387-4c06-46cc-bb8c-38ddb73724bc","Type":"ContainerStarted","Data":"270f7fc22133a7bdd407bb2a07305bfd0810cc2bfe8997314a77c0c53a725cd1"} Jul 27 20:57:58.784895 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:58.784490 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-khcmr" Jul 27 20:57:58.804508 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:57:58.804449 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-khcmr" podStartSLOduration=1.679781081 podStartE2EDuration="3.804436556s" totalImagesPullingTime="2.124655475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:57:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:57:58.803067256 +0000 UTC m=+407.326184980" watchObservedRunningTime="2026-07-27 20:57:58.804436556 +0000 UTC m=+407.327554277" Jul 27 20:58:02.365218 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:02.365185 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-cz96d"] Jul 27 20:58:02.368492 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:02.368474 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cz96d" Jul 27 20:58:02.371607 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:02.371586 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 27 20:58:02.371746 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:02.371729 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-t6w6q\"" Jul 27 20:58:02.371819 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:02.371770 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 27 20:58:02.378810 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:02.378786 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-cz96d"] Jul 27 20:58:02.471493 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:02.471457 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/508970d0-2590-4bd5-bfc4-74cc2dc5013a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-cz96d\" (UID: \"508970d0-2590-4bd5-bfc4-74cc2dc5013a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cz96d" Jul 27 20:58:02.471493 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:02.471492 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/508970d0-2590-4bd5-bfc4-74cc2dc5013a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cz96d\" (UID: \"508970d0-2590-4bd5-bfc4-74cc2dc5013a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cz96d" Jul 27 20:58:02.471688 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:02.471524 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkkgr\" (UniqueName: \"kubernetes.io/projected/508970d0-2590-4bd5-bfc4-74cc2dc5013a-kube-api-access-jkkgr\") pod \"kuadrant-console-plugin-66b5774985-cz96d\" (UID: \"508970d0-2590-4bd5-bfc4-74cc2dc5013a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cz96d" Jul 27 20:58:02.572879 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:02.572842 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/508970d0-2590-4bd5-bfc4-74cc2dc5013a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-cz96d\" (UID: \"508970d0-2590-4bd5-bfc4-74cc2dc5013a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cz96d" Jul 27 20:58:02.573045 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:02.572882 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/508970d0-2590-4bd5-bfc4-74cc2dc5013a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cz96d\" (UID: \"508970d0-2590-4bd5-bfc4-74cc2dc5013a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cz96d" Jul 27 20:58:02.573045 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:02.572933 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkkgr\" (UniqueName: \"kubernetes.io/projected/508970d0-2590-4bd5-bfc4-74cc2dc5013a-kube-api-access-jkkgr\") pod \"kuadrant-console-plugin-66b5774985-cz96d\" (UID: \"508970d0-2590-4bd5-bfc4-74cc2dc5013a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cz96d" Jul 27 20:58:02.573118 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:58:02.573076 2565 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 27 20:58:02.573166 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:58:02.573156 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/508970d0-2590-4bd5-bfc4-74cc2dc5013a-plugin-serving-cert podName:508970d0-2590-4bd5-bfc4-74cc2dc5013a nodeName:}" failed. No retries permitted until 2026-07-27 20:58:03.073137333 +0000 UTC m=+411.596255038 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/508970d0-2590-4bd5-bfc4-74cc2dc5013a-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-cz96d" (UID: "508970d0-2590-4bd5-bfc4-74cc2dc5013a") : secret "plugin-serving-cert" not found Jul 27 20:58:02.573652 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:02.573625 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/508970d0-2590-4bd5-bfc4-74cc2dc5013a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-cz96d\" (UID: \"508970d0-2590-4bd5-bfc4-74cc2dc5013a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cz96d" Jul 27 20:58:02.594224 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:02.594192 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkkgr\" (UniqueName: \"kubernetes.io/projected/508970d0-2590-4bd5-bfc4-74cc2dc5013a-kube-api-access-jkkgr\") pod \"kuadrant-console-plugin-66b5774985-cz96d\" (UID: \"508970d0-2590-4bd5-bfc4-74cc2dc5013a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cz96d" Jul 27 20:58:03.077466 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:03.077412 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/508970d0-2590-4bd5-bfc4-74cc2dc5013a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cz96d\" (UID: \"508970d0-2590-4bd5-bfc4-74cc2dc5013a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cz96d" Jul 27 20:58:03.080045 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:03.080017 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/508970d0-2590-4bd5-bfc4-74cc2dc5013a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cz96d\" (UID: \"508970d0-2590-4bd5-bfc4-74cc2dc5013a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cz96d" Jul 27 20:58:03.278365 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:03.278313 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cz96d" Jul 27 20:58:03.405697 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:03.405665 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-cz96d"] Jul 27 20:58:03.407178 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:58:03.407149 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod508970d0_2590_4bd5_bfc4_74cc2dc5013a.slice/crio-123e8af4d7fd300a2fd4bdc0e62473f5a1ab6e913a3e2d82d3389833f6d1ad48 WatchSource:0}: Error finding container 123e8af4d7fd300a2fd4bdc0e62473f5a1ab6e913a3e2d82d3389833f6d1ad48: Status 404 returned error can't find the container with id 123e8af4d7fd300a2fd4bdc0e62473f5a1ab6e913a3e2d82d3389833f6d1ad48 Jul 27 20:58:03.806866 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:03.806828 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cz96d" event={"ID":"508970d0-2590-4bd5-bfc4-74cc2dc5013a","Type":"ContainerStarted","Data":"123e8af4d7fd300a2fd4bdc0e62473f5a1ab6e913a3e2d82d3389833f6d1ad48"} Jul 27 20:58:09.791492 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:09.791462 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-khcmr" Jul 27 20:58:24.917096 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:24.917056 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cz96d" event={"ID":"508970d0-2590-4bd5-bfc4-74cc2dc5013a","Type":"ContainerStarted","Data":"392fe9d58752122fcc2adc4d77d4df07ec9b3817d9950fdbdd045524cafa10e6"} Jul 27 20:58:24.935431 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:24.935376 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cz96d" podStartSLOduration=2.089204949 podStartE2EDuration="22.935360708s" totalImagesPullingTime="20.846155759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:58:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:58:24.932900507 +0000 UTC m=+433.456018243" watchObservedRunningTime="2026-07-27 20:58:24.935360708 +0000 UTC m=+433.458478441" Jul 27 20:58:44.064261 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.064178 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5"] Jul 27 20:58:44.177803 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.177766 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5"] Jul 27 20:58:44.177965 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.177878 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.180732 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.180710 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-hwmpn\"" Jul 27 20:58:44.337410 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.337339 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2647a034-e168-465b-aad6-8f7a13f91580-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.337410 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.337379 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2647a034-e168-465b-aad6-8f7a13f91580-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.337410 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.337398 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2647a034-e168-465b-aad6-8f7a13f91580-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.337614 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.337458 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2647a034-e168-465b-aad6-8f7a13f91580-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.337614 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.337518 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2647a034-e168-465b-aad6-8f7a13f91580-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.337614 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.337534 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2647a034-e168-465b-aad6-8f7a13f91580-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.337614 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.337553 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2647a034-e168-465b-aad6-8f7a13f91580-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.337734 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.337620 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrfdd\" (UniqueName: \"kubernetes.io/projected/2647a034-e168-465b-aad6-8f7a13f91580-kube-api-access-qrfdd\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.337734 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.337667 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2647a034-e168-465b-aad6-8f7a13f91580-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.438288 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.438214 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2647a034-e168-465b-aad6-8f7a13f91580-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.438477 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.438300 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2647a034-e168-465b-aad6-8f7a13f91580-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.438477 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.438342 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2647a034-e168-465b-aad6-8f7a13f91580-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.438477 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.438391 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrfdd\" (UniqueName: \"kubernetes.io/projected/2647a034-e168-465b-aad6-8f7a13f91580-kube-api-access-qrfdd\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.438477 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.438409 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2647a034-e168-465b-aad6-8f7a13f91580-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.438477 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.438431 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2647a034-e168-465b-aad6-8f7a13f91580-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.438477 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.438461 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2647a034-e168-465b-aad6-8f7a13f91580-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.438729 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.438488 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2647a034-e168-465b-aad6-8f7a13f91580-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.438729 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.438519 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2647a034-e168-465b-aad6-8f7a13f91580-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.438855 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.438796 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2647a034-e168-465b-aad6-8f7a13f91580-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.438855 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.438840 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2647a034-e168-465b-aad6-8f7a13f91580-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.438944 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.438881 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2647a034-e168-465b-aad6-8f7a13f91580-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.439045 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.439027 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2647a034-e168-465b-aad6-8f7a13f91580-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.439217 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.439200 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2647a034-e168-465b-aad6-8f7a13f91580-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.440935 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.440915 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2647a034-e168-465b-aad6-8f7a13f91580-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.441221 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.441201 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2647a034-e168-465b-aad6-8f7a13f91580-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.446738 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.446713 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2647a034-e168-465b-aad6-8f7a13f91580-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.446896 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.446876 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrfdd\" (UniqueName: \"kubernetes.io/projected/2647a034-e168-465b-aad6-8f7a13f91580-kube-api-access-qrfdd\") pod \"maas-default-gateway-openshift-default-78bb78ff78-pt9n5\" (UID: \"2647a034-e168-465b-aad6-8f7a13f91580\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.488079 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.488035 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:44.650525 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.650500 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5"] Jul 27 20:58:44.652639 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:58:44.652612 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2647a034_e168_465b_aad6_8f7a13f91580.slice/crio-3569473fff00afe3a0b73b6b80e0fecbcbf30e485a1000b4ee332ec38060e7fb WatchSource:0}: Error finding container 3569473fff00afe3a0b73b6b80e0fecbcbf30e485a1000b4ee332ec38060e7fb: Status 404 returned error can't find the container with id 3569473fff00afe3a0b73b6b80e0fecbcbf30e485a1000b4ee332ec38060e7fb Jul 27 20:58:44.655334 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.655298 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 20:58:44.655398 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.655383 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 20:58:44.655439 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:44.655423 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 20:58:45.009948 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:45.009907 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" event={"ID":"2647a034-e168-465b-aad6-8f7a13f91580","Type":"ContainerStarted","Data":"39edce6b99945a73325e183f6fe8064dafdad93fc95f38a0e1a5f561a039e9c0"} Jul 27 20:58:45.010159 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:45.009966 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" event={"ID":"2647a034-e168-465b-aad6-8f7a13f91580","Type":"ContainerStarted","Data":"3569473fff00afe3a0b73b6b80e0fecbcbf30e485a1000b4ee332ec38060e7fb"} Jul 27 20:58:45.034021 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:45.033960 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" podStartSLOduration=1.033940608 podStartE2EDuration="1.033940608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:58:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:58:45.033310123 +0000 UTC m=+453.556427846" watchObservedRunningTime="2026-07-27 20:58:45.033940608 +0000 UTC m=+453.557058330" Jul 27 20:58:45.488553 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:45.488518 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:45.493288 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:45.493229 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:46.014362 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:46.014327 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:46.015258 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:46.015222 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 20:58:46.915008 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:46.914973 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-6nvwp"] Jul 27 20:58:46.979780 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:46.979741 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6nvwp"] Jul 27 20:58:46.979954 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:46.979863 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6nvwp" Jul 27 20:58:46.982804 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:46.982779 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bghfh\"" Jul 27 20:58:47.041447 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:47.041412 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-855v4"] Jul 27 20:58:47.061377 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:47.061342 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhzsr\" (UniqueName: \"kubernetes.io/projected/72bcc498-6c95-474b-872e-5f80b3bf1bd4-kube-api-access-hhzsr\") pod \"authorino-58cd955675-6nvwp\" (UID: \"72bcc498-6c95-474b-872e-5f80b3bf1bd4\") " pod="kuadrant-system/authorino-58cd955675-6nvwp" Jul 27 20:58:47.078523 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:47.078481 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-855v4"] Jul 27 20:58:47.078671 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:47.078582 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-855v4" Jul 27 20:58:47.162657 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:47.162616 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq869\" (UniqueName: \"kubernetes.io/projected/80bde95c-eb22-4055-b614-f128e4213624-kube-api-access-sq869\") pod \"authorino-75d556bd77-855v4\" (UID: \"80bde95c-eb22-4055-b614-f128e4213624\") " pod="kuadrant-system/authorino-75d556bd77-855v4" Jul 27 20:58:47.162852 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:47.162750 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhzsr\" (UniqueName: \"kubernetes.io/projected/72bcc498-6c95-474b-872e-5f80b3bf1bd4-kube-api-access-hhzsr\") pod \"authorino-58cd955675-6nvwp\" (UID: \"72bcc498-6c95-474b-872e-5f80b3bf1bd4\") " pod="kuadrant-system/authorino-58cd955675-6nvwp" Jul 27 20:58:47.171711 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:47.171638 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhzsr\" (UniqueName: \"kubernetes.io/projected/72bcc498-6c95-474b-872e-5f80b3bf1bd4-kube-api-access-hhzsr\") pod \"authorino-58cd955675-6nvwp\" (UID: \"72bcc498-6c95-474b-872e-5f80b3bf1bd4\") " pod="kuadrant-system/authorino-58cd955675-6nvwp" Jul 27 20:58:47.263896 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:47.263857 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq869\" (UniqueName: \"kubernetes.io/projected/80bde95c-eb22-4055-b614-f128e4213624-kube-api-access-sq869\") pod \"authorino-75d556bd77-855v4\" (UID: \"80bde95c-eb22-4055-b614-f128e4213624\") " pod="kuadrant-system/authorino-75d556bd77-855v4" Jul 27 20:58:47.273039 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:47.272998 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq869\" (UniqueName: \"kubernetes.io/projected/80bde95c-eb22-4055-b614-f128e4213624-kube-api-access-sq869\") pod \"authorino-75d556bd77-855v4\" (UID: \"80bde95c-eb22-4055-b614-f128e4213624\") " pod="kuadrant-system/authorino-75d556bd77-855v4" Jul 27 20:58:47.288873 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:47.288846 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6nvwp" Jul 27 20:58:47.387339 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:47.387305 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-855v4" Jul 27 20:58:47.462596 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:47.462544 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6nvwp"] Jul 27 20:58:47.464435 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:58:47.464295 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72bcc498_6c95_474b_872e_5f80b3bf1bd4.slice/crio-303ab5e2294eda646ce456c570eb9ba4478127e77860cd31872bc95add448b9d WatchSource:0}: Error finding container 303ab5e2294eda646ce456c570eb9ba4478127e77860cd31872bc95add448b9d: Status 404 returned error can't find the container with id 303ab5e2294eda646ce456c570eb9ba4478127e77860cd31872bc95add448b9d Jul 27 20:58:47.542649 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:47.542597 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-855v4"] Jul 27 20:58:47.544654 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:58:47.544628 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80bde95c_eb22_4055_b614_f128e4213624.slice/crio-d1668c0b977a400a6445baa1ed6d0410096a00fea8c415a1293e552e460cfba8 WatchSource:0}: Error finding container d1668c0b977a400a6445baa1ed6d0410096a00fea8c415a1293e552e460cfba8: Status 404 returned error can't find the container with id d1668c0b977a400a6445baa1ed6d0410096a00fea8c415a1293e552e460cfba8 Jul 27 20:58:48.024992 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:48.024958 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-855v4" event={"ID":"80bde95c-eb22-4055-b614-f128e4213624","Type":"ContainerStarted","Data":"d1668c0b977a400a6445baa1ed6d0410096a00fea8c415a1293e552e460cfba8"} Jul 27 20:58:48.026156 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:48.026125 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6nvwp" event={"ID":"72bcc498-6c95-474b-872e-5f80b3bf1bd4","Type":"ContainerStarted","Data":"303ab5e2294eda646ce456c570eb9ba4478127e77860cd31872bc95add448b9d"} Jul 27 20:58:53.061497 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:53.061448 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6nvwp" event={"ID":"72bcc498-6c95-474b-872e-5f80b3bf1bd4","Type":"ContainerStarted","Data":"b0fc3fb283e6ddd6f992d814a5e5c266c75a15c8539deba5294386fe3ac83592"} Jul 27 20:58:53.062958 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:53.062931 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-855v4" event={"ID":"80bde95c-eb22-4055-b614-f128e4213624","Type":"ContainerStarted","Data":"86df59381489c4ee6438701feadbdc5e40f3aa0387d3d96de258bbb83fbe4f0a"} Jul 27 20:58:53.077733 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:53.077686 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-6nvwp" podStartSLOduration=2.443226527 podStartE2EDuration="7.077670727s" totalImagesPullingTime="4.6344442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:58:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:58:53.075850335 +0000 UTC m=+461.598968056" watchObservedRunningTime="2026-07-27 20:58:53.077670727 +0000 UTC m=+461.600788452" Jul 27 20:58:53.105498 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:53.105437 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-855v4" podStartSLOduration=1.564689058 podStartE2EDuration="6.105422277s" totalImagesPullingTime="4.540733219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:58:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:58:53.105382126 +0000 UTC m=+461.628499848" watchObservedRunningTime="2026-07-27 20:58:53.105422277 +0000 UTC m=+461.628539999" Jul 27 20:58:53.115358 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:53.115329 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6nvwp"] Jul 27 20:58:55.071673 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:55.071634 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-6nvwp" podUID="72bcc498-6c95-474b-872e-5f80b3bf1bd4" containerName="authorino" containerID="cri-o://b0fc3fb283e6ddd6f992d814a5e5c266c75a15c8539deba5294386fe3ac83592" gracePeriod=30 Jul 27 20:58:55.311936 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:55.311908 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6nvwp" Jul 27 20:58:55.438967 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:55.438934 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhzsr\" (UniqueName: \"kubernetes.io/projected/72bcc498-6c95-474b-872e-5f80b3bf1bd4-kube-api-access-hhzsr\") pod \"72bcc498-6c95-474b-872e-5f80b3bf1bd4\" (UID: \"72bcc498-6c95-474b-872e-5f80b3bf1bd4\") " Jul 27 20:58:55.441416 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:55.441387 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72bcc498-6c95-474b-872e-5f80b3bf1bd4-kube-api-access-hhzsr" (OuterVolumeSpecName: "kube-api-access-hhzsr") pod "72bcc498-6c95-474b-872e-5f80b3bf1bd4" (UID: "72bcc498-6c95-474b-872e-5f80b3bf1bd4"). InnerVolumeSpecName "kube-api-access-hhzsr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 20:58:55.540002 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:55.539962 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhzsr\" (UniqueName: \"kubernetes.io/projected/72bcc498-6c95-474b-872e-5f80b3bf1bd4-kube-api-access-hhzsr\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:58:56.077545 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:56.077513 2565 generic.go:356] "Generic (PLEG): container finished" podID="72bcc498-6c95-474b-872e-5f80b3bf1bd4" containerID="b0fc3fb283e6ddd6f992d814a5e5c266c75a15c8539deba5294386fe3ac83592" exitCode=0 Jul 27 20:58:56.077974 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:56.077567 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6nvwp" Jul 27 20:58:56.077974 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:56.077601 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6nvwp" event={"ID":"72bcc498-6c95-474b-872e-5f80b3bf1bd4","Type":"ContainerDied","Data":"b0fc3fb283e6ddd6f992d814a5e5c266c75a15c8539deba5294386fe3ac83592"} Jul 27 20:58:56.077974 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:56.077645 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6nvwp" event={"ID":"72bcc498-6c95-474b-872e-5f80b3bf1bd4","Type":"ContainerDied","Data":"303ab5e2294eda646ce456c570eb9ba4478127e77860cd31872bc95add448b9d"} Jul 27 20:58:56.077974 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:56.077663 2565 scope.go:117] "RemoveContainer" containerID="b0fc3fb283e6ddd6f992d814a5e5c266c75a15c8539deba5294386fe3ac83592" Jul 27 20:58:56.086309 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:56.086294 2565 scope.go:117] "RemoveContainer" containerID="b0fc3fb283e6ddd6f992d814a5e5c266c75a15c8539deba5294386fe3ac83592" Jul 27 20:58:56.086535 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:58:56.086515 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0fc3fb283e6ddd6f992d814a5e5c266c75a15c8539deba5294386fe3ac83592\": container with ID starting with b0fc3fb283e6ddd6f992d814a5e5c266c75a15c8539deba5294386fe3ac83592 not found: ID does not exist" containerID="b0fc3fb283e6ddd6f992d814a5e5c266c75a15c8539deba5294386fe3ac83592" Jul 27 20:58:56.086604 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:56.086546 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0fc3fb283e6ddd6f992d814a5e5c266c75a15c8539deba5294386fe3ac83592"} err="failed to get container status \"b0fc3fb283e6ddd6f992d814a5e5c266c75a15c8539deba5294386fe3ac83592\": rpc error: code = NotFound desc = could not find container \"b0fc3fb283e6ddd6f992d814a5e5c266c75a15c8539deba5294386fe3ac83592\": container with ID starting with b0fc3fb283e6ddd6f992d814a5e5c266c75a15c8539deba5294386fe3ac83592 not found: ID does not exist" Jul 27 20:58:56.093749 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:56.093727 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6nvwp"] Jul 27 20:58:56.097709 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:56.097687 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-6nvwp"] Jul 27 20:58:58.052331 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:58:58.052302 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72bcc498-6c95-474b-872e-5f80b3bf1bd4" path="/var/lib/kubelet/pods/72bcc498-6c95-474b-872e-5f80b3bf1bd4/volumes" Jul 27 20:59:20.343317 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.343284 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-clbs2"] Jul 27 20:59:20.343781 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.343763 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72bcc498-6c95-474b-872e-5f80b3bf1bd4" containerName="authorino" Jul 27 20:59:20.343913 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.343785 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="72bcc498-6c95-474b-872e-5f80b3bf1bd4" containerName="authorino" Jul 27 20:59:20.343913 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.343880 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="72bcc498-6c95-474b-872e-5f80b3bf1bd4" containerName="authorino" Jul 27 20:59:20.349506 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.349487 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-clbs2" Jul 27 20:59:20.355754 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.355726 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-clbs2"] Jul 27 20:59:20.450893 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.450856 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gz89\" (UniqueName: \"kubernetes.io/projected/394d21eb-e0d9-4482-a7d5-d4c90d23fbd7-kube-api-access-8gz89\") pod \"authorino-5d4b67c446-clbs2\" (UID: \"394d21eb-e0d9-4482-a7d5-d4c90d23fbd7\") " pod="kuadrant-system/authorino-5d4b67c446-clbs2" Jul 27 20:59:20.527264 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.527220 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-clbs2"] Jul 27 20:59:20.527488 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:59:20.527469 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-8gz89], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-clbs2" podUID="394d21eb-e0d9-4482-a7d5-d4c90d23fbd7" Jul 27 20:59:20.551583 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.551551 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gz89\" (UniqueName: \"kubernetes.io/projected/394d21eb-e0d9-4482-a7d5-d4c90d23fbd7-kube-api-access-8gz89\") pod \"authorino-5d4b67c446-clbs2\" (UID: \"394d21eb-e0d9-4482-a7d5-d4c90d23fbd7\") " pod="kuadrant-system/authorino-5d4b67c446-clbs2" Jul 27 20:59:20.560443 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.560417 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gz89\" (UniqueName: \"kubernetes.io/projected/394d21eb-e0d9-4482-a7d5-d4c90d23fbd7-kube-api-access-8gz89\") pod \"authorino-5d4b67c446-clbs2\" (UID: \"394d21eb-e0d9-4482-a7d5-d4c90d23fbd7\") " pod="kuadrant-system/authorino-5d4b67c446-clbs2" Jul 27 20:59:20.566698 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.566673 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-9zjxz"] Jul 27 20:59:20.573040 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.573016 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-9zjxz" Jul 27 20:59:20.575870 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.575852 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 27 20:59:20.578671 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.578645 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-9zjxz"] Jul 27 20:59:20.615306 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.615226 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-9zjxz"] Jul 27 20:59:20.615493 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:59:20.615475 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-s7b82 tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-s7b82 tls-cert]: context canceled" pod="kuadrant-system/authorino-76877994-9zjxz" podUID="05caea66-b2eb-4a4c-9be0-62989c26b0cd" Jul 27 20:59:20.647036 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.647012 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-66d6cd795-6644c"] Jul 27 20:59:20.651466 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.651445 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66d6cd795-6644c" Jul 27 20:59:20.652025 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.651995 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7b82\" (UniqueName: \"kubernetes.io/projected/05caea66-b2eb-4a4c-9be0-62989c26b0cd-kube-api-access-s7b82\") pod \"authorino-76877994-9zjxz\" (UID: \"05caea66-b2eb-4a4c-9be0-62989c26b0cd\") " pod="kuadrant-system/authorino-76877994-9zjxz" Jul 27 20:59:20.652106 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.652046 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/05caea66-b2eb-4a4c-9be0-62989c26b0cd-tls-cert\") pod \"authorino-76877994-9zjxz\" (UID: \"05caea66-b2eb-4a4c-9be0-62989c26b0cd\") " pod="kuadrant-system/authorino-76877994-9zjxz" Jul 27 20:59:20.658211 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.658059 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66d6cd795-6644c"] Jul 27 20:59:20.753203 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.753160 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs7cl\" (UniqueName: \"kubernetes.io/projected/a269ed3c-fea8-4df7-ac3d-b1529401a2f0-kube-api-access-xs7cl\") pod \"authorino-66d6cd795-6644c\" (UID: \"a269ed3c-fea8-4df7-ac3d-b1529401a2f0\") " pod="kuadrant-system/authorino-66d6cd795-6644c" Jul 27 20:59:20.753203 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.753207 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/05caea66-b2eb-4a4c-9be0-62989c26b0cd-tls-cert\") pod \"authorino-76877994-9zjxz\" (UID: \"05caea66-b2eb-4a4c-9be0-62989c26b0cd\") " pod="kuadrant-system/authorino-76877994-9zjxz" Jul 27 20:59:20.753452 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.753307 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a269ed3c-fea8-4df7-ac3d-b1529401a2f0-tls-cert\") pod \"authorino-66d6cd795-6644c\" (UID: \"a269ed3c-fea8-4df7-ac3d-b1529401a2f0\") " pod="kuadrant-system/authorino-66d6cd795-6644c" Jul 27 20:59:20.753452 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.753342 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7b82\" (UniqueName: \"kubernetes.io/projected/05caea66-b2eb-4a4c-9be0-62989c26b0cd-kube-api-access-s7b82\") pod \"authorino-76877994-9zjxz\" (UID: \"05caea66-b2eb-4a4c-9be0-62989c26b0cd\") " pod="kuadrant-system/authorino-76877994-9zjxz" Jul 27 20:59:20.755745 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.755719 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/05caea66-b2eb-4a4c-9be0-62989c26b0cd-tls-cert\") pod \"authorino-76877994-9zjxz\" (UID: \"05caea66-b2eb-4a4c-9be0-62989c26b0cd\") " pod="kuadrant-system/authorino-76877994-9zjxz" Jul 27 20:59:20.762129 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.762107 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7b82\" (UniqueName: \"kubernetes.io/projected/05caea66-b2eb-4a4c-9be0-62989c26b0cd-kube-api-access-s7b82\") pod \"authorino-76877994-9zjxz\" (UID: \"05caea66-b2eb-4a4c-9be0-62989c26b0cd\") " pod="kuadrant-system/authorino-76877994-9zjxz" Jul 27 20:59:20.854684 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.854648 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a269ed3c-fea8-4df7-ac3d-b1529401a2f0-tls-cert\") pod \"authorino-66d6cd795-6644c\" (UID: \"a269ed3c-fea8-4df7-ac3d-b1529401a2f0\") " pod="kuadrant-system/authorino-66d6cd795-6644c" Jul 27 20:59:20.854848 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.854713 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs7cl\" (UniqueName: \"kubernetes.io/projected/a269ed3c-fea8-4df7-ac3d-b1529401a2f0-kube-api-access-xs7cl\") pod \"authorino-66d6cd795-6644c\" (UID: \"a269ed3c-fea8-4df7-ac3d-b1529401a2f0\") " pod="kuadrant-system/authorino-66d6cd795-6644c" Jul 27 20:59:20.857184 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.857152 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a269ed3c-fea8-4df7-ac3d-b1529401a2f0-tls-cert\") pod \"authorino-66d6cd795-6644c\" (UID: \"a269ed3c-fea8-4df7-ac3d-b1529401a2f0\") " pod="kuadrant-system/authorino-66d6cd795-6644c" Jul 27 20:59:20.862740 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.862718 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs7cl\" (UniqueName: \"kubernetes.io/projected/a269ed3c-fea8-4df7-ac3d-b1529401a2f0-kube-api-access-xs7cl\") pod \"authorino-66d6cd795-6644c\" (UID: \"a269ed3c-fea8-4df7-ac3d-b1529401a2f0\") " pod="kuadrant-system/authorino-66d6cd795-6644c" Jul 27 20:59:20.961068 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:20.961028 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66d6cd795-6644c" Jul 27 20:59:21.086778 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:21.086753 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66d6cd795-6644c"] Jul 27 20:59:21.088592 ip-10-0-131-232 kubenswrapper[2565]: W0727 20:59:21.088569 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda269ed3c_fea8_4df7_ac3d_b1529401a2f0.slice/crio-f1e2f21f4c47ff30284650f47dd3ac7204e2547d07b11f14a805982b372c817e WatchSource:0}: Error finding container f1e2f21f4c47ff30284650f47dd3ac7204e2547d07b11f14a805982b372c817e: Status 404 returned error can't find the container with id f1e2f21f4c47ff30284650f47dd3ac7204e2547d07b11f14a805982b372c817e Jul 27 20:59:21.195547 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:21.195508 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66d6cd795-6644c" event={"ID":"a269ed3c-fea8-4df7-ac3d-b1529401a2f0","Type":"ContainerStarted","Data":"f1e2f21f4c47ff30284650f47dd3ac7204e2547d07b11f14a805982b372c817e"} Jul 27 20:59:21.195547 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:21.195545 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-clbs2" Jul 27 20:59:21.195738 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:21.195545 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-9zjxz" Jul 27 20:59:21.202448 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:21.202428 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-clbs2" Jul 27 20:59:21.204495 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:21.204477 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-9zjxz" Jul 27 20:59:21.257442 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:21.257377 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gz89\" (UniqueName: \"kubernetes.io/projected/394d21eb-e0d9-4482-a7d5-d4c90d23fbd7-kube-api-access-8gz89\") pod \"394d21eb-e0d9-4482-a7d5-d4c90d23fbd7\" (UID: \"394d21eb-e0d9-4482-a7d5-d4c90d23fbd7\") " Jul 27 20:59:21.257442 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:21.257405 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7b82\" (UniqueName: \"kubernetes.io/projected/05caea66-b2eb-4a4c-9be0-62989c26b0cd-kube-api-access-s7b82\") pod \"05caea66-b2eb-4a4c-9be0-62989c26b0cd\" (UID: \"05caea66-b2eb-4a4c-9be0-62989c26b0cd\") " Jul 27 20:59:21.257607 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:21.257469 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/05caea66-b2eb-4a4c-9be0-62989c26b0cd-tls-cert\") pod \"05caea66-b2eb-4a4c-9be0-62989c26b0cd\" (UID: \"05caea66-b2eb-4a4c-9be0-62989c26b0cd\") " Jul 27 20:59:21.259544 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:21.259511 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05caea66-b2eb-4a4c-9be0-62989c26b0cd-kube-api-access-s7b82" (OuterVolumeSpecName: "kube-api-access-s7b82") pod "05caea66-b2eb-4a4c-9be0-62989c26b0cd" (UID: "05caea66-b2eb-4a4c-9be0-62989c26b0cd"). InnerVolumeSpecName "kube-api-access-s7b82". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 20:59:21.259644 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:21.259565 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05caea66-b2eb-4a4c-9be0-62989c26b0cd-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "05caea66-b2eb-4a4c-9be0-62989c26b0cd" (UID: "05caea66-b2eb-4a4c-9be0-62989c26b0cd"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 20:59:21.259644 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:21.259589 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/394d21eb-e0d9-4482-a7d5-d4c90d23fbd7-kube-api-access-8gz89" (OuterVolumeSpecName: "kube-api-access-8gz89") pod "394d21eb-e0d9-4482-a7d5-d4c90d23fbd7" (UID: "394d21eb-e0d9-4482-a7d5-d4c90d23fbd7"). InnerVolumeSpecName "kube-api-access-8gz89". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 20:59:21.358845 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:21.358809 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gz89\" (UniqueName: \"kubernetes.io/projected/394d21eb-e0d9-4482-a7d5-d4c90d23fbd7-kube-api-access-8gz89\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:59:21.358845 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:21.358837 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7b82\" (UniqueName: \"kubernetes.io/projected/05caea66-b2eb-4a4c-9be0-62989c26b0cd-kube-api-access-s7b82\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:59:21.358845 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:21.358847 2565 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/05caea66-b2eb-4a4c-9be0-62989c26b0cd-tls-cert\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:59:22.201587 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:22.201543 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66d6cd795-6644c" event={"ID":"a269ed3c-fea8-4df7-ac3d-b1529401a2f0","Type":"ContainerStarted","Data":"c2e311081bf35971b300806292a47e024eda88d7ef14c40191d903ec4903400b"} Jul 27 20:59:22.201587 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:22.201567 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-9zjxz" Jul 27 20:59:22.201587 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:22.201576 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-clbs2" Jul 27 20:59:22.219530 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:22.219480 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-66d6cd795-6644c" podStartSLOduration=1.825870624 podStartE2EDuration="2.21946807s" totalImagesPullingTime="393.597446ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:59:22.217581344 +0000 UTC m=+490.740699066" watchObservedRunningTime="2026-07-27 20:59:22.21946807 +0000 UTC m=+490.742585792" Jul 27 20:59:22.242209 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:22.242183 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-9zjxz"] Jul 27 20:59:22.246102 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:22.246075 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-855v4"] Jul 27 20:59:22.246284 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:22.246252 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-855v4" podUID="80bde95c-eb22-4055-b614-f128e4213624" containerName="authorino" containerID="cri-o://86df59381489c4ee6438701feadbdc5e40f3aa0387d3d96de258bbb83fbe4f0a" gracePeriod=30 Jul 27 20:59:22.250151 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:22.250073 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-9zjxz"] Jul 27 20:59:22.272433 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:22.272404 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-clbs2"] Jul 27 20:59:22.274363 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:22.274345 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-clbs2"] Jul 27 20:59:22.501365 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:22.501339 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-855v4" Jul 27 20:59:22.569230 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:22.569201 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sq869\" (UniqueName: \"kubernetes.io/projected/80bde95c-eb22-4055-b614-f128e4213624-kube-api-access-sq869\") pod \"80bde95c-eb22-4055-b614-f128e4213624\" (UID: \"80bde95c-eb22-4055-b614-f128e4213624\") " Jul 27 20:59:22.571329 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:22.571300 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80bde95c-eb22-4055-b614-f128e4213624-kube-api-access-sq869" (OuterVolumeSpecName: "kube-api-access-sq869") pod "80bde95c-eb22-4055-b614-f128e4213624" (UID: "80bde95c-eb22-4055-b614-f128e4213624"). InnerVolumeSpecName "kube-api-access-sq869". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 20:59:22.670449 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:22.670422 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq869\" (UniqueName: \"kubernetes.io/projected/80bde95c-eb22-4055-b614-f128e4213624-kube-api-access-sq869\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 20:59:23.206723 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:23.206691 2565 generic.go:356] "Generic (PLEG): container finished" podID="80bde95c-eb22-4055-b614-f128e4213624" containerID="86df59381489c4ee6438701feadbdc5e40f3aa0387d3d96de258bbb83fbe4f0a" exitCode=0 Jul 27 20:59:23.206890 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:23.206735 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-855v4" Jul 27 20:59:23.206890 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:23.206768 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-855v4" event={"ID":"80bde95c-eb22-4055-b614-f128e4213624","Type":"ContainerDied","Data":"86df59381489c4ee6438701feadbdc5e40f3aa0387d3d96de258bbb83fbe4f0a"} Jul 27 20:59:23.206890 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:23.206818 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-855v4" event={"ID":"80bde95c-eb22-4055-b614-f128e4213624","Type":"ContainerDied","Data":"d1668c0b977a400a6445baa1ed6d0410096a00fea8c415a1293e552e460cfba8"} Jul 27 20:59:23.206890 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:23.206841 2565 scope.go:117] "RemoveContainer" containerID="86df59381489c4ee6438701feadbdc5e40f3aa0387d3d96de258bbb83fbe4f0a" Jul 27 20:59:23.219604 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:23.219578 2565 scope.go:117] "RemoveContainer" containerID="86df59381489c4ee6438701feadbdc5e40f3aa0387d3d96de258bbb83fbe4f0a" Jul 27 20:59:23.219954 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:59:23.219935 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86df59381489c4ee6438701feadbdc5e40f3aa0387d3d96de258bbb83fbe4f0a\": container with ID starting with 86df59381489c4ee6438701feadbdc5e40f3aa0387d3d96de258bbb83fbe4f0a not found: ID does not exist" containerID="86df59381489c4ee6438701feadbdc5e40f3aa0387d3d96de258bbb83fbe4f0a" Jul 27 20:59:23.220015 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:23.219964 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86df59381489c4ee6438701feadbdc5e40f3aa0387d3d96de258bbb83fbe4f0a"} err="failed to get container status \"86df59381489c4ee6438701feadbdc5e40f3aa0387d3d96de258bbb83fbe4f0a\": rpc error: code = NotFound desc = could not find container \"86df59381489c4ee6438701feadbdc5e40f3aa0387d3d96de258bbb83fbe4f0a\": container with ID starting with 86df59381489c4ee6438701feadbdc5e40f3aa0387d3d96de258bbb83fbe4f0a not found: ID does not exist" Jul 27 20:59:23.232423 ip-10-0-131-232 kubenswrapper[2565]: E0727 20:59:23.232392 2565 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80bde95c_eb22_4055_b614_f128e4213624.slice/crio-d1668c0b977a400a6445baa1ed6d0410096a00fea8c415a1293e552e460cfba8\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80bde95c_eb22_4055_b614_f128e4213624.slice\": RecentStats: unable to find data in memory cache]" Jul 27 20:59:23.236959 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:23.236934 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-855v4"] Jul 27 20:59:23.243601 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:23.243578 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-855v4"] Jul 27 20:59:24.051735 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:24.051702 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05caea66-b2eb-4a4c-9be0-62989c26b0cd" path="/var/lib/kubelet/pods/05caea66-b2eb-4a4c-9be0-62989c26b0cd/volumes" Jul 27 20:59:24.052106 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:24.051916 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="394d21eb-e0d9-4482-a7d5-d4c90d23fbd7" path="/var/lib/kubelet/pods/394d21eb-e0d9-4482-a7d5-d4c90d23fbd7/volumes" Jul 27 20:59:24.052106 ip-10-0-131-232 kubenswrapper[2565]: I0727 20:59:24.052090 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80bde95c-eb22-4055-b614-f128e4213624" path="/var/lib/kubelet/pods/80bde95c-eb22-4055-b614-f128e4213624/volumes" Jul 27 21:00:13.441370 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:13.441336 2565 generic.go:356] "Generic (PLEG): container finished" podID="2647a034-e168-465b-aad6-8f7a13f91580" containerID="39edce6b99945a73325e183f6fe8064dafdad93fc95f38a0e1a5f561a039e9c0" exitCode=0 Jul 27 21:00:13.441824 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:13.441393 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" event={"ID":"2647a034-e168-465b-aad6-8f7a13f91580","Type":"ContainerDied","Data":"39edce6b99945a73325e183f6fe8064dafdad93fc95f38a0e1a5f561a039e9c0"} Jul 27 21:00:13.441824 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:13.441697 2565 scope.go:117] "RemoveContainer" containerID="39edce6b99945a73325e183f6fe8064dafdad93fc95f38a0e1a5f561a039e9c0" Jul 27 21:00:13.442615 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:13.442589 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:00:13.442676 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:13.442653 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:00:13.442719 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:13.442690 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:00:14.448248 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:14.448201 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" event={"ID":"2647a034-e168-465b-aad6-8f7a13f91580","Type":"ContainerStarted","Data":"f39c993127f9911c25b2d2d7bb86dfb089ea496c0994caf2e5b34f0629c97f11"} Jul 27 21:00:14.488483 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:14.488449 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 21:00:14.489859 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:14.489825 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-pt9n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 27 21:00:14.490003 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:14.489899 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" podUID="2647a034-e168-465b-aad6-8f7a13f91580" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 27 21:00:15.488773 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:15.488736 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-pt9n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 27 21:00:15.489163 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:15.488789 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" podUID="2647a034-e168-465b-aad6-8f7a13f91580" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 27 21:00:16.014317 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:16.014284 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 21:00:16.488997 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:16.488957 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-pt9n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 27 21:00:16.489506 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:16.489025 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" podUID="2647a034-e168-465b-aad6-8f7a13f91580" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 27 21:00:17.489088 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:17.489048 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-pt9n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 27 21:00:17.489575 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:17.489106 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" podUID="2647a034-e168-465b-aad6-8f7a13f91580" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 27 21:00:18.488838 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:18.488806 2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-pt9n5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 27 21:00:18.489014 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:18.488870 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" podUID="2647a034-e168-465b-aad6-8f7a13f91580" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 27 21:00:19.493098 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:19.493068 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 21:00:19.494842 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:19.494810 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-pt9n5" Jul 27 21:00:42.421025 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.420962 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp"] Jul 27 21:00:42.421512 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.421483 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80bde95c-eb22-4055-b614-f128e4213624" containerName="authorino" Jul 27 21:00:42.421512 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.421501 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="80bde95c-eb22-4055-b614-f128e4213624" containerName="authorino" Jul 27 21:00:42.421597 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.421584 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="80bde95c-eb22-4055-b614-f128e4213624" containerName="authorino" Jul 27 21:00:42.424368 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.424346 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.428245 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.428207 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 27 21:00:42.428382 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.428263 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 27 21:00:42.428382 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.428278 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qcv5g\"" Jul 27 21:00:42.428382 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.428347 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 27 21:00:42.432511 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.432490 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp"] Jul 27 21:00:42.527464 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.527429 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/489e7645-73f9-4487-b3bb-8fe2d316ff02-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t2twp\" (UID: \"489e7645-73f9-4487-b3bb-8fe2d316ff02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.527464 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.527473 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/489e7645-73f9-4487-b3bb-8fe2d316ff02-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t2twp\" (UID: \"489e7645-73f9-4487-b3bb-8fe2d316ff02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.527696 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.527501 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/489e7645-73f9-4487-b3bb-8fe2d316ff02-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t2twp\" (UID: \"489e7645-73f9-4487-b3bb-8fe2d316ff02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.527696 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.527585 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/489e7645-73f9-4487-b3bb-8fe2d316ff02-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t2twp\" (UID: \"489e7645-73f9-4487-b3bb-8fe2d316ff02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.527696 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.527623 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6vcm\" (UniqueName: \"kubernetes.io/projected/489e7645-73f9-4487-b3bb-8fe2d316ff02-kube-api-access-h6vcm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t2twp\" (UID: \"489e7645-73f9-4487-b3bb-8fe2d316ff02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.527811 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.527701 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/489e7645-73f9-4487-b3bb-8fe2d316ff02-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t2twp\" (UID: \"489e7645-73f9-4487-b3bb-8fe2d316ff02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.629092 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.629051 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/489e7645-73f9-4487-b3bb-8fe2d316ff02-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t2twp\" (UID: \"489e7645-73f9-4487-b3bb-8fe2d316ff02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.629092 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.629100 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6vcm\" (UniqueName: \"kubernetes.io/projected/489e7645-73f9-4487-b3bb-8fe2d316ff02-kube-api-access-h6vcm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t2twp\" (UID: \"489e7645-73f9-4487-b3bb-8fe2d316ff02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.629404 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.629151 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/489e7645-73f9-4487-b3bb-8fe2d316ff02-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t2twp\" (UID: \"489e7645-73f9-4487-b3bb-8fe2d316ff02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.629404 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.629181 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/489e7645-73f9-4487-b3bb-8fe2d316ff02-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t2twp\" (UID: \"489e7645-73f9-4487-b3bb-8fe2d316ff02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.629404 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.629196 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/489e7645-73f9-4487-b3bb-8fe2d316ff02-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t2twp\" (UID: \"489e7645-73f9-4487-b3bb-8fe2d316ff02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.629404 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.629356 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/489e7645-73f9-4487-b3bb-8fe2d316ff02-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t2twp\" (UID: \"489e7645-73f9-4487-b3bb-8fe2d316ff02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.629637 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.629506 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/489e7645-73f9-4487-b3bb-8fe2d316ff02-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t2twp\" (UID: \"489e7645-73f9-4487-b3bb-8fe2d316ff02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.629637 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.629569 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/489e7645-73f9-4487-b3bb-8fe2d316ff02-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t2twp\" (UID: \"489e7645-73f9-4487-b3bb-8fe2d316ff02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.629637 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.629626 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/489e7645-73f9-4487-b3bb-8fe2d316ff02-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t2twp\" (UID: \"489e7645-73f9-4487-b3bb-8fe2d316ff02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.631619 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.631600 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/489e7645-73f9-4487-b3bb-8fe2d316ff02-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t2twp\" (UID: \"489e7645-73f9-4487-b3bb-8fe2d316ff02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.631793 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.631775 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/489e7645-73f9-4487-b3bb-8fe2d316ff02-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t2twp\" (UID: \"489e7645-73f9-4487-b3bb-8fe2d316ff02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.637293 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.637268 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6vcm\" (UniqueName: \"kubernetes.io/projected/489e7645-73f9-4487-b3bb-8fe2d316ff02-kube-api-access-h6vcm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t2twp\" (UID: \"489e7645-73f9-4487-b3bb-8fe2d316ff02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.736168 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.736058 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:42.870309 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:42.870285 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp"] Jul 27 21:00:42.872036 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:00:42.872014 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod489e7645_73f9_4487_b3bb_8fe2d316ff02.slice/crio-01c973fb14665859b12330b6fdef92cdc1d11655853ce7105cec0514c6f72990 WatchSource:0}: Error finding container 01c973fb14665859b12330b6fdef92cdc1d11655853ce7105cec0514c6f72990: Status 404 returned error can't find the container with id 01c973fb14665859b12330b6fdef92cdc1d11655853ce7105cec0514c6f72990 Jul 27 21:00:43.587680 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:43.587638 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" event={"ID":"489e7645-73f9-4487-b3bb-8fe2d316ff02","Type":"ContainerStarted","Data":"01c973fb14665859b12330b6fdef92cdc1d11655853ce7105cec0514c6f72990"} Jul 27 21:00:47.606664 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:47.606627 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" event={"ID":"489e7645-73f9-4487-b3bb-8fe2d316ff02","Type":"ContainerStarted","Data":"a523f2d41875a381223ee28ad262804791935d1a642f3069e04464904db43b5d"} Jul 27 21:00:47.624660 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:47.624612 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" podStartSLOduration=1.909766838 podStartE2EDuration="5.624598263s" totalImagesPullingTime="3.714831425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:00:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:00:47.623515748 +0000 UTC m=+576.146633469" watchObservedRunningTime="2026-07-27 21:00:47.624598263 +0000 UTC m=+576.147715985" Jul 27 21:00:52.736599 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:52.736556 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:52.736966 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:52.736627 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:52.748983 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:52.748958 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:00:53.648213 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:00:53.648182 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t2twp" Jul 27 21:01:05.424556 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.424526 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld"] Jul 27 21:01:05.427410 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.427395 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.430216 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.430195 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 27 21:01:05.436056 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.436031 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld"] Jul 27 21:01:05.519888 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.519856 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/788b3609-5f93-4615-9d60-1d87132a8836-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld\" (UID: \"788b3609-5f93-4615-9d60-1d87132a8836\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.520056 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.519905 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/788b3609-5f93-4615-9d60-1d87132a8836-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld\" (UID: \"788b3609-5f93-4615-9d60-1d87132a8836\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.520056 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.519962 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/788b3609-5f93-4615-9d60-1d87132a8836-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld\" (UID: \"788b3609-5f93-4615-9d60-1d87132a8836\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.520056 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.520011 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/788b3609-5f93-4615-9d60-1d87132a8836-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld\" (UID: \"788b3609-5f93-4615-9d60-1d87132a8836\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.520056 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.520031 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7xbq\" (UniqueName: \"kubernetes.io/projected/788b3609-5f93-4615-9d60-1d87132a8836-kube-api-access-w7xbq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld\" (UID: \"788b3609-5f93-4615-9d60-1d87132a8836\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.520188 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.520062 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/788b3609-5f93-4615-9d60-1d87132a8836-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld\" (UID: \"788b3609-5f93-4615-9d60-1d87132a8836\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.621201 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.621163 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/788b3609-5f93-4615-9d60-1d87132a8836-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld\" (UID: \"788b3609-5f93-4615-9d60-1d87132a8836\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.621382 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.621222 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/788b3609-5f93-4615-9d60-1d87132a8836-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld\" (UID: \"788b3609-5f93-4615-9d60-1d87132a8836\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.621382 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.621277 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/788b3609-5f93-4615-9d60-1d87132a8836-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld\" (UID: \"788b3609-5f93-4615-9d60-1d87132a8836\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.621382 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.621324 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/788b3609-5f93-4615-9d60-1d87132a8836-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld\" (UID: \"788b3609-5f93-4615-9d60-1d87132a8836\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.621382 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.621366 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/788b3609-5f93-4615-9d60-1d87132a8836-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld\" (UID: \"788b3609-5f93-4615-9d60-1d87132a8836\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.621550 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.621390 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7xbq\" (UniqueName: \"kubernetes.io/projected/788b3609-5f93-4615-9d60-1d87132a8836-kube-api-access-w7xbq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld\" (UID: \"788b3609-5f93-4615-9d60-1d87132a8836\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.621669 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.621647 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/788b3609-5f93-4615-9d60-1d87132a8836-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld\" (UID: \"788b3609-5f93-4615-9d60-1d87132a8836\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.621749 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.621726 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/788b3609-5f93-4615-9d60-1d87132a8836-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld\" (UID: \"788b3609-5f93-4615-9d60-1d87132a8836\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.621846 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.621822 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/788b3609-5f93-4615-9d60-1d87132a8836-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld\" (UID: \"788b3609-5f93-4615-9d60-1d87132a8836\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.623625 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.623601 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/788b3609-5f93-4615-9d60-1d87132a8836-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld\" (UID: \"788b3609-5f93-4615-9d60-1d87132a8836\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.623882 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.623864 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/788b3609-5f93-4615-9d60-1d87132a8836-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld\" (UID: \"788b3609-5f93-4615-9d60-1d87132a8836\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.629614 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.629591 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7xbq\" (UniqueName: \"kubernetes.io/projected/788b3609-5f93-4615-9d60-1d87132a8836-kube-api-access-w7xbq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld\" (UID: \"788b3609-5f93-4615-9d60-1d87132a8836\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.737682 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.737602 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:05.870502 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:05.870477 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld"] Jul 27 21:01:05.872148 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:01:05.872120 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod788b3609_5f93_4615_9d60_1d87132a8836.slice/crio-815facad832e1bad689b558e3ba52c2457f94cce1d83aa1199a3605acf88344b WatchSource:0}: Error finding container 815facad832e1bad689b558e3ba52c2457f94cce1d83aa1199a3605acf88344b: Status 404 returned error can't find the container with id 815facad832e1bad689b558e3ba52c2457f94cce1d83aa1199a3605acf88344b Jul 27 21:01:06.699214 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:06.699173 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" event={"ID":"788b3609-5f93-4615-9d60-1d87132a8836","Type":"ContainerStarted","Data":"87ce6b298757084714aa56c5e24c97f4353683b6b1080969174536a9c8d058b4"} Jul 27 21:01:06.699598 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:06.699251 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" event={"ID":"788b3609-5f93-4615-9d60-1d87132a8836","Type":"ContainerStarted","Data":"815facad832e1bad689b558e3ba52c2457f94cce1d83aa1199a3605acf88344b"} Jul 27 21:01:06.717382 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:06.717340 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" podStartSLOduration=1.024119484 podStartE2EDuration="1.717324542s" totalImagesPullingTime="693.205058ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:01:06.716573731 +0000 UTC m=+595.239691454" watchObservedRunningTime="2026-07-27 21:01:06.717324542 +0000 UTC m=+595.240442263" Jul 27 21:01:15.738564 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:15.738527 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:15.738924 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:15.738588 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:15.751222 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:15.751199 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:01:15.762486 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:01:15.762468 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld" Jul 27 21:02:27.866368 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:27.866329 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8658494974-jpmxq"] Jul 27 21:02:27.869719 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:27.869702 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8658494974-jpmxq" Jul 27 21:02:27.880799 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:27.880775 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8658494974-jpmxq"] Jul 27 21:02:28.047857 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:28.047823 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb494\" (UniqueName: \"kubernetes.io/projected/44a27f06-a463-48c6-85f3-a888e14dd522-kube-api-access-tb494\") pod \"authorino-8658494974-jpmxq\" (UID: \"44a27f06-a463-48c6-85f3-a888e14dd522\") " pod="kuadrant-system/authorino-8658494974-jpmxq" Jul 27 21:02:28.047857 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:28.047862 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/44a27f06-a463-48c6-85f3-a888e14dd522-tls-cert\") pod \"authorino-8658494974-jpmxq\" (UID: \"44a27f06-a463-48c6-85f3-a888e14dd522\") " pod="kuadrant-system/authorino-8658494974-jpmxq" Jul 27 21:02:28.149136 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:28.149054 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb494\" (UniqueName: \"kubernetes.io/projected/44a27f06-a463-48c6-85f3-a888e14dd522-kube-api-access-tb494\") pod \"authorino-8658494974-jpmxq\" (UID: \"44a27f06-a463-48c6-85f3-a888e14dd522\") " pod="kuadrant-system/authorino-8658494974-jpmxq" Jul 27 21:02:28.149136 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:28.149104 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/44a27f06-a463-48c6-85f3-a888e14dd522-tls-cert\") pod \"authorino-8658494974-jpmxq\" (UID: \"44a27f06-a463-48c6-85f3-a888e14dd522\") " pod="kuadrant-system/authorino-8658494974-jpmxq" Jul 27 21:02:28.151618 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:28.151598 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/44a27f06-a463-48c6-85f3-a888e14dd522-tls-cert\") pod \"authorino-8658494974-jpmxq\" (UID: \"44a27f06-a463-48c6-85f3-a888e14dd522\") " pod="kuadrant-system/authorino-8658494974-jpmxq" Jul 27 21:02:28.159129 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:28.159104 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb494\" (UniqueName: \"kubernetes.io/projected/44a27f06-a463-48c6-85f3-a888e14dd522-kube-api-access-tb494\") pod \"authorino-8658494974-jpmxq\" (UID: \"44a27f06-a463-48c6-85f3-a888e14dd522\") " pod="kuadrant-system/authorino-8658494974-jpmxq" Jul 27 21:02:28.178204 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:28.178176 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8658494974-jpmxq" Jul 27 21:02:28.308707 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:28.308684 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8658494974-jpmxq"] Jul 27 21:02:28.310301 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:02:28.310274 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44a27f06_a463_48c6_85f3_a888e14dd522.slice/crio-cb5baea2f381c1183d993aeb307b934e1704ab13f42630a00c3debb08d25c54b WatchSource:0}: Error finding container cb5baea2f381c1183d993aeb307b934e1704ab13f42630a00c3debb08d25c54b: Status 404 returned error can't find the container with id cb5baea2f381c1183d993aeb307b934e1704ab13f42630a00c3debb08d25c54b Jul 27 21:02:28.311635 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:28.311618 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 21:02:29.093971 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:29.093926 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8658494974-jpmxq" event={"ID":"44a27f06-a463-48c6-85f3-a888e14dd522","Type":"ContainerStarted","Data":"d6c690672fa78b68166fe3163e12ba1720e8c3080687b48e05635e00cd6eb317"} Jul 27 21:02:29.094468 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:29.093986 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8658494974-jpmxq" event={"ID":"44a27f06-a463-48c6-85f3-a888e14dd522","Type":"ContainerStarted","Data":"cb5baea2f381c1183d993aeb307b934e1704ab13f42630a00c3debb08d25c54b"} Jul 27 21:02:29.111851 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:29.111462 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8658494974-jpmxq" podStartSLOduration=1.465437654 podStartE2EDuration="2.1114424s" totalImagesPullingTime="646.004746ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:02:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:02:29.111322352 +0000 UTC m=+677.634440075" watchObservedRunningTime="2026-07-27 21:02:29.1114424 +0000 UTC m=+677.634560125" Jul 27 21:02:29.140785 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:29.140746 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66d6cd795-6644c"] Jul 27 21:02:29.140982 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:29.140955 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-66d6cd795-6644c" podUID="a269ed3c-fea8-4df7-ac3d-b1529401a2f0" containerName="authorino" containerID="cri-o://c2e311081bf35971b300806292a47e024eda88d7ef14c40191d903ec4903400b" gracePeriod=30 Jul 27 21:02:29.396833 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:29.396805 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66d6cd795-6644c" Jul 27 21:02:29.561367 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:29.561281 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xs7cl\" (UniqueName: \"kubernetes.io/projected/a269ed3c-fea8-4df7-ac3d-b1529401a2f0-kube-api-access-xs7cl\") pod \"a269ed3c-fea8-4df7-ac3d-b1529401a2f0\" (UID: \"a269ed3c-fea8-4df7-ac3d-b1529401a2f0\") " Jul 27 21:02:29.561367 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:29.561366 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a269ed3c-fea8-4df7-ac3d-b1529401a2f0-tls-cert\") pod \"a269ed3c-fea8-4df7-ac3d-b1529401a2f0\" (UID: \"a269ed3c-fea8-4df7-ac3d-b1529401a2f0\") " Jul 27 21:02:29.563627 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:29.563589 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a269ed3c-fea8-4df7-ac3d-b1529401a2f0-kube-api-access-xs7cl" (OuterVolumeSpecName: "kube-api-access-xs7cl") pod "a269ed3c-fea8-4df7-ac3d-b1529401a2f0" (UID: "a269ed3c-fea8-4df7-ac3d-b1529401a2f0"). InnerVolumeSpecName "kube-api-access-xs7cl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:02:29.572891 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:29.572863 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a269ed3c-fea8-4df7-ac3d-b1529401a2f0-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "a269ed3c-fea8-4df7-ac3d-b1529401a2f0" (UID: "a269ed3c-fea8-4df7-ac3d-b1529401a2f0"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 21:02:29.662433 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:29.662388 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xs7cl\" (UniqueName: \"kubernetes.io/projected/a269ed3c-fea8-4df7-ac3d-b1529401a2f0-kube-api-access-xs7cl\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:02:29.662433 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:29.662433 2565 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a269ed3c-fea8-4df7-ac3d-b1529401a2f0-tls-cert\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:02:30.099605 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:30.099573 2565 generic.go:356] "Generic (PLEG): container finished" podID="a269ed3c-fea8-4df7-ac3d-b1529401a2f0" containerID="c2e311081bf35971b300806292a47e024eda88d7ef14c40191d903ec4903400b" exitCode=0 Jul 27 21:02:30.100003 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:30.099643 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66d6cd795-6644c" Jul 27 21:02:30.100003 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:30.099659 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66d6cd795-6644c" event={"ID":"a269ed3c-fea8-4df7-ac3d-b1529401a2f0","Type":"ContainerDied","Data":"c2e311081bf35971b300806292a47e024eda88d7ef14c40191d903ec4903400b"} Jul 27 21:02:30.100003 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:30.099701 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66d6cd795-6644c" event={"ID":"a269ed3c-fea8-4df7-ac3d-b1529401a2f0","Type":"ContainerDied","Data":"f1e2f21f4c47ff30284650f47dd3ac7204e2547d07b11f14a805982b372c817e"} Jul 27 21:02:30.100003 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:30.099721 2565 scope.go:117] "RemoveContainer" containerID="c2e311081bf35971b300806292a47e024eda88d7ef14c40191d903ec4903400b" Jul 27 21:02:30.108534 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:30.108518 2565 scope.go:117] "RemoveContainer" containerID="c2e311081bf35971b300806292a47e024eda88d7ef14c40191d903ec4903400b" Jul 27 21:02:30.108787 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:02:30.108769 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2e311081bf35971b300806292a47e024eda88d7ef14c40191d903ec4903400b\": container with ID starting with c2e311081bf35971b300806292a47e024eda88d7ef14c40191d903ec4903400b not found: ID does not exist" containerID="c2e311081bf35971b300806292a47e024eda88d7ef14c40191d903ec4903400b" Jul 27 21:02:30.108852 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:30.108796 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2e311081bf35971b300806292a47e024eda88d7ef14c40191d903ec4903400b"} err="failed to get container status \"c2e311081bf35971b300806292a47e024eda88d7ef14c40191d903ec4903400b\": rpc error: code = NotFound desc = could not find container \"c2e311081bf35971b300806292a47e024eda88d7ef14c40191d903ec4903400b\": container with ID starting with c2e311081bf35971b300806292a47e024eda88d7ef14c40191d903ec4903400b not found: ID does not exist" Jul 27 21:02:30.147937 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:30.147901 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66d6cd795-6644c"] Jul 27 21:02:30.155578 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:30.155554 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-66d6cd795-6644c"] Jul 27 21:02:32.052054 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:02:32.052022 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a269ed3c-fea8-4df7-ac3d-b1529401a2f0" path="/var/lib/kubelet/pods/a269ed3c-fea8-4df7-ac3d-b1529401a2f0/volumes" Jul 27 21:04:18.911002 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:18.910967 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-55784848bf-7lw5d"] Jul 27 21:04:18.911461 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:18.911302 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a269ed3c-fea8-4df7-ac3d-b1529401a2f0" containerName="authorino" Jul 27 21:04:18.911461 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:18.911313 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a269ed3c-fea8-4df7-ac3d-b1529401a2f0" containerName="authorino" Jul 27 21:04:18.911461 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:18.911380 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a269ed3c-fea8-4df7-ac3d-b1529401a2f0" containerName="authorino" Jul 27 21:04:18.913580 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:18.913565 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55784848bf-7lw5d" Jul 27 21:04:18.917524 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:18.917501 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kdkqw\"" Jul 27 21:04:18.917524 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:18.917517 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 27 21:04:18.917721 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:18.917503 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 27 21:04:18.923724 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:18.923701 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55784848bf-7lw5d"] Jul 27 21:04:19.006907 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:19.006868 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hghbr\" (UniqueName: \"kubernetes.io/projected/19a1482a-a196-4c5c-8026-d34ecbf4e74c-kube-api-access-hghbr\") pod \"maas-controller-55784848bf-7lw5d\" (UID: \"19a1482a-a196-4c5c-8026-d34ecbf4e74c\") " pod="opendatahub/maas-controller-55784848bf-7lw5d" Jul 27 21:04:19.007078 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:19.006997 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/19a1482a-a196-4c5c-8026-d34ecbf4e74c-webhook-cert\") pod \"maas-controller-55784848bf-7lw5d\" (UID: \"19a1482a-a196-4c5c-8026-d34ecbf4e74c\") " pod="opendatahub/maas-controller-55784848bf-7lw5d" Jul 27 21:04:19.107643 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:19.107608 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hghbr\" (UniqueName: \"kubernetes.io/projected/19a1482a-a196-4c5c-8026-d34ecbf4e74c-kube-api-access-hghbr\") pod \"maas-controller-55784848bf-7lw5d\" (UID: \"19a1482a-a196-4c5c-8026-d34ecbf4e74c\") " pod="opendatahub/maas-controller-55784848bf-7lw5d" Jul 27 21:04:19.107801 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:19.107687 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/19a1482a-a196-4c5c-8026-d34ecbf4e74c-webhook-cert\") pod \"maas-controller-55784848bf-7lw5d\" (UID: \"19a1482a-a196-4c5c-8026-d34ecbf4e74c\") " pod="opendatahub/maas-controller-55784848bf-7lw5d" Jul 27 21:04:19.110357 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:19.110323 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/19a1482a-a196-4c5c-8026-d34ecbf4e74c-webhook-cert\") pod \"maas-controller-55784848bf-7lw5d\" (UID: \"19a1482a-a196-4c5c-8026-d34ecbf4e74c\") " pod="opendatahub/maas-controller-55784848bf-7lw5d" Jul 27 21:04:19.117329 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:19.117308 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hghbr\" (UniqueName: \"kubernetes.io/projected/19a1482a-a196-4c5c-8026-d34ecbf4e74c-kube-api-access-hghbr\") pod \"maas-controller-55784848bf-7lw5d\" (UID: \"19a1482a-a196-4c5c-8026-d34ecbf4e74c\") " pod="opendatahub/maas-controller-55784848bf-7lw5d" Jul 27 21:04:19.223358 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:19.223327 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55784848bf-7lw5d" Jul 27 21:04:19.357480 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:19.357453 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55784848bf-7lw5d"] Jul 27 21:04:19.358701 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:04:19.358674 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a1482a_a196_4c5c_8026_d34ecbf4e74c.slice/crio-d09736158ad74475fa865dc3972454a47bc60d46629ae9e14027e840713acf44 WatchSource:0}: Error finding container d09736158ad74475fa865dc3972454a47bc60d46629ae9e14027e840713acf44: Status 404 returned error can't find the container with id d09736158ad74475fa865dc3972454a47bc60d46629ae9e14027e840713acf44 Jul 27 21:04:19.612800 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:19.612720 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55784848bf-7lw5d" event={"ID":"19a1482a-a196-4c5c-8026-d34ecbf4e74c","Type":"ContainerStarted","Data":"d09736158ad74475fa865dc3972454a47bc60d46629ae9e14027e840713acf44"} Jul 27 21:04:22.628621 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:22.628589 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55784848bf-7lw5d" event={"ID":"19a1482a-a196-4c5c-8026-d34ecbf4e74c","Type":"ContainerStarted","Data":"81d77addd404a6bae042a59eb471ac3238c4b570c93a085b3133d141d2cb6917"} Jul 27 21:04:22.629004 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:22.628726 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-55784848bf-7lw5d" Jul 27 21:04:22.648684 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:22.648636 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-55784848bf-7lw5d" podStartSLOduration=2.28803325 podStartE2EDuration="4.648622045s" totalImagesPullingTime="2.360588795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:04:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:04:22.645657646 +0000 UTC m=+791.168775364" watchObservedRunningTime="2026-07-27 21:04:22.648622045 +0000 UTC m=+791.171739766" Jul 27 21:04:33.635544 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:33.635466 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-55784848bf-7lw5d" Jul 27 21:04:38.277404 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:38.277369 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55784848bf-7lw5d"] Jul 27 21:04:38.277800 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:38.277596 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-55784848bf-7lw5d" podUID="19a1482a-a196-4c5c-8026-d34ecbf4e74c" containerName="manager" containerID="cri-o://81d77addd404a6bae042a59eb471ac3238c4b570c93a085b3133d141d2cb6917" gracePeriod=30 Jul 27 21:04:38.529289 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:38.529215 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55784848bf-7lw5d" Jul 27 21:04:38.572552 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:38.572522 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/19a1482a-a196-4c5c-8026-d34ecbf4e74c-webhook-cert\") pod \"19a1482a-a196-4c5c-8026-d34ecbf4e74c\" (UID: \"19a1482a-a196-4c5c-8026-d34ecbf4e74c\") " Jul 27 21:04:38.572813 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:38.572789 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hghbr\" (UniqueName: \"kubernetes.io/projected/19a1482a-a196-4c5c-8026-d34ecbf4e74c-kube-api-access-hghbr\") pod \"19a1482a-a196-4c5c-8026-d34ecbf4e74c\" (UID: \"19a1482a-a196-4c5c-8026-d34ecbf4e74c\") " Jul 27 21:04:38.575147 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:38.575120 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19a1482a-a196-4c5c-8026-d34ecbf4e74c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "19a1482a-a196-4c5c-8026-d34ecbf4e74c" (UID: "19a1482a-a196-4c5c-8026-d34ecbf4e74c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 21:04:38.575550 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:38.575521 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19a1482a-a196-4c5c-8026-d34ecbf4e74c-kube-api-access-hghbr" (OuterVolumeSpecName: "kube-api-access-hghbr") pod "19a1482a-a196-4c5c-8026-d34ecbf4e74c" (UID: "19a1482a-a196-4c5c-8026-d34ecbf4e74c"). InnerVolumeSpecName "kube-api-access-hghbr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:04:38.674005 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:38.673965 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hghbr\" (UniqueName: \"kubernetes.io/projected/19a1482a-a196-4c5c-8026-d34ecbf4e74c-kube-api-access-hghbr\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:04:38.674005 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:38.674000 2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/19a1482a-a196-4c5c-8026-d34ecbf4e74c-webhook-cert\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:04:38.704564 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:38.704533 2565 generic.go:356] "Generic (PLEG): container finished" podID="19a1482a-a196-4c5c-8026-d34ecbf4e74c" containerID="81d77addd404a6bae042a59eb471ac3238c4b570c93a085b3133d141d2cb6917" exitCode=0 Jul 27 21:04:38.704695 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:38.704582 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55784848bf-7lw5d" Jul 27 21:04:38.704695 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:38.704598 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55784848bf-7lw5d" event={"ID":"19a1482a-a196-4c5c-8026-d34ecbf4e74c","Type":"ContainerDied","Data":"81d77addd404a6bae042a59eb471ac3238c4b570c93a085b3133d141d2cb6917"} Jul 27 21:04:38.704695 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:38.704642 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55784848bf-7lw5d" event={"ID":"19a1482a-a196-4c5c-8026-d34ecbf4e74c","Type":"ContainerDied","Data":"d09736158ad74475fa865dc3972454a47bc60d46629ae9e14027e840713acf44"} Jul 27 21:04:38.704695 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:38.704664 2565 scope.go:117] "RemoveContainer" containerID="81d77addd404a6bae042a59eb471ac3238c4b570c93a085b3133d141d2cb6917" Jul 27 21:04:38.714095 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:38.714077 2565 scope.go:117] "RemoveContainer" containerID="81d77addd404a6bae042a59eb471ac3238c4b570c93a085b3133d141d2cb6917" Jul 27 21:04:38.714370 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:04:38.714350 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81d77addd404a6bae042a59eb471ac3238c4b570c93a085b3133d141d2cb6917\": container with ID starting with 81d77addd404a6bae042a59eb471ac3238c4b570c93a085b3133d141d2cb6917 not found: ID does not exist" containerID="81d77addd404a6bae042a59eb471ac3238c4b570c93a085b3133d141d2cb6917" Jul 27 21:04:38.714418 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:38.714380 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81d77addd404a6bae042a59eb471ac3238c4b570c93a085b3133d141d2cb6917"} err="failed to get container status \"81d77addd404a6bae042a59eb471ac3238c4b570c93a085b3133d141d2cb6917\": rpc error: code = NotFound desc = could not find container \"81d77addd404a6bae042a59eb471ac3238c4b570c93a085b3133d141d2cb6917\": container with ID starting with 81d77addd404a6bae042a59eb471ac3238c4b570c93a085b3133d141d2cb6917 not found: ID does not exist" Jul 27 21:04:38.726360 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:38.726336 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55784848bf-7lw5d"] Jul 27 21:04:38.728595 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:38.728575 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-55784848bf-7lw5d"] Jul 27 21:04:39.504760 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:39.504727 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-55784848bf-82r7n"] Jul 27 21:04:39.505164 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:39.505049 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19a1482a-a196-4c5c-8026-d34ecbf4e74c" containerName="manager" Jul 27 21:04:39.505164 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:39.505060 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="19a1482a-a196-4c5c-8026-d34ecbf4e74c" containerName="manager" Jul 27 21:04:39.505164 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:39.505119 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="19a1482a-a196-4c5c-8026-d34ecbf4e74c" containerName="manager" Jul 27 21:04:39.508439 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:39.508415 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55784848bf-82r7n" Jul 27 21:04:39.512355 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:39.512336 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 27 21:04:39.512464 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:39.512336 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 27 21:04:39.512464 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:39.512373 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kdkqw\"" Jul 27 21:04:39.516146 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:39.515964 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55784848bf-82r7n"] Jul 27 21:04:39.583140 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:39.583104 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sjbf\" (UniqueName: \"kubernetes.io/projected/06301a5d-f175-4d9e-99d7-056173c19ae5-kube-api-access-9sjbf\") pod \"maas-controller-55784848bf-82r7n\" (UID: \"06301a5d-f175-4d9e-99d7-056173c19ae5\") " pod="opendatahub/maas-controller-55784848bf-82r7n" Jul 27 21:04:39.583140 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:39.583141 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06301a5d-f175-4d9e-99d7-056173c19ae5-webhook-cert\") pod \"maas-controller-55784848bf-82r7n\" (UID: \"06301a5d-f175-4d9e-99d7-056173c19ae5\") " pod="opendatahub/maas-controller-55784848bf-82r7n" Jul 27 21:04:39.684413 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:39.684381 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sjbf\" (UniqueName: \"kubernetes.io/projected/06301a5d-f175-4d9e-99d7-056173c19ae5-kube-api-access-9sjbf\") pod \"maas-controller-55784848bf-82r7n\" (UID: \"06301a5d-f175-4d9e-99d7-056173c19ae5\") " pod="opendatahub/maas-controller-55784848bf-82r7n" Jul 27 21:04:39.684413 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:39.684416 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06301a5d-f175-4d9e-99d7-056173c19ae5-webhook-cert\") pod \"maas-controller-55784848bf-82r7n\" (UID: \"06301a5d-f175-4d9e-99d7-056173c19ae5\") " pod="opendatahub/maas-controller-55784848bf-82r7n" Jul 27 21:04:39.687057 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:39.687029 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06301a5d-f175-4d9e-99d7-056173c19ae5-webhook-cert\") pod \"maas-controller-55784848bf-82r7n\" (UID: \"06301a5d-f175-4d9e-99d7-056173c19ae5\") " pod="opendatahub/maas-controller-55784848bf-82r7n" Jul 27 21:04:39.693036 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:39.693016 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sjbf\" (UniqueName: \"kubernetes.io/projected/06301a5d-f175-4d9e-99d7-056173c19ae5-kube-api-access-9sjbf\") pod \"maas-controller-55784848bf-82r7n\" (UID: \"06301a5d-f175-4d9e-99d7-056173c19ae5\") " pod="opendatahub/maas-controller-55784848bf-82r7n" Jul 27 21:04:39.819774 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:39.819702 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55784848bf-82r7n" Jul 27 21:04:39.957331 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:39.957306 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55784848bf-82r7n"] Jul 27 21:04:39.960124 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:04:39.960099 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06301a5d_f175_4d9e_99d7_056173c19ae5.slice/crio-6c491e20a15e9feca00f6955796cb3872103795da1bf3565414dd16e9ac1fb1e WatchSource:0}: Error finding container 6c491e20a15e9feca00f6955796cb3872103795da1bf3565414dd16e9ac1fb1e: Status 404 returned error can't find the container with id 6c491e20a15e9feca00f6955796cb3872103795da1bf3565414dd16e9ac1fb1e Jul 27 21:04:40.052608 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:40.052565 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19a1482a-a196-4c5c-8026-d34ecbf4e74c" path="/var/lib/kubelet/pods/19a1482a-a196-4c5c-8026-d34ecbf4e74c/volumes" Jul 27 21:04:40.716515 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:40.716480 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55784848bf-82r7n" event={"ID":"06301a5d-f175-4d9e-99d7-056173c19ae5","Type":"ContainerStarted","Data":"cc78e1441e2ae6b87859e9cc85ba06c711cd95d14429a2d834639434bf7fa66b"} Jul 27 21:04:40.716913 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:40.716540 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55784848bf-82r7n" event={"ID":"06301a5d-f175-4d9e-99d7-056173c19ae5","Type":"ContainerStarted","Data":"6c491e20a15e9feca00f6955796cb3872103795da1bf3565414dd16e9ac1fb1e"} Jul 27 21:04:40.716913 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:40.716630 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-55784848bf-82r7n" Jul 27 21:04:40.736672 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:40.736623 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-55784848bf-82r7n" podStartSLOduration=1.30459582 podStartE2EDuration="1.736607345s" totalImagesPullingTime="432.011525ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:04:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:04:40.735743336 +0000 UTC m=+809.258861071" watchObservedRunningTime="2026-07-27 21:04:40.736607345 +0000 UTC m=+809.259725066" Jul 27 21:04:51.724639 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:04:51.724602 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-55784848bf-82r7n" Jul 27 21:13:49.621381 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.621346 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf"] Jul 27 21:13:49.625209 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.625170 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:49.628026 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.628006 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-379c72fa-kserve-self-signed-certs\"" Jul 27 21:13:49.634769 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.634746 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf"] Jul 27 21:13:49.710987 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.710953 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-dshm\") pod \"e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:49.710987 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.710996 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-tmp-dir\") pod \"e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:49.711279 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.711076 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-home\") pod \"e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:49.711279 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.711131 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-model-cache\") pod \"e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:49.711279 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.711156 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgcz9\" (UniqueName: \"kubernetes.io/projected/53674d5c-bbff-4bd3-833f-f0ed51d675c6-kube-api-access-xgcz9\") pod \"e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:49.711279 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.711182 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53674d5c-bbff-4bd3-833f-f0ed51d675c6-tls-certs\") pod \"e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:49.812363 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.812325 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-home\") pod \"e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:49.812547 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.812378 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-model-cache\") pod \"e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:49.812547 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.812499 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgcz9\" (UniqueName: \"kubernetes.io/projected/53674d5c-bbff-4bd3-833f-f0ed51d675c6-kube-api-access-xgcz9\") pod \"e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:49.812669 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.812544 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53674d5c-bbff-4bd3-833f-f0ed51d675c6-tls-certs\") pod \"e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:49.812669 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.812627 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-dshm\") pod \"e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:49.812669 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.812657 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-tmp-dir\") pod \"e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:49.812802 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.812777 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-model-cache\") pod \"e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:49.812835 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.812812 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-home\") pod \"e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:49.813027 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.813004 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-tmp-dir\") pod \"e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:49.815032 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.815002 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-dshm\") pod \"e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:49.815268 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.815254 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53674d5c-bbff-4bd3-833f-f0ed51d675c6-tls-certs\") pod \"e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:49.820702 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.820683 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgcz9\" (UniqueName: \"kubernetes.io/projected/53674d5c-bbff-4bd3-833f-f0ed51d675c6-kube-api-access-xgcz9\") pod \"e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:49.935588 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:49.935558 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:50.070956 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:50.070923 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf"] Jul 27 21:13:50.072302 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:13:50.072268 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53674d5c_bbff_4bd3_833f_f0ed51d675c6.slice/crio-7132ffdd85c60e365cd06370bf7afe1ed91860f3a717fef9465bb9b152245b04 WatchSource:0}: Error finding container 7132ffdd85c60e365cd06370bf7afe1ed91860f3a717fef9465bb9b152245b04: Status 404 returned error can't find the container with id 7132ffdd85c60e365cd06370bf7afe1ed91860f3a717fef9465bb9b152245b04 Jul 27 21:13:50.372101 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:50.372011 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" event={"ID":"53674d5c-bbff-4bd3-833f-f0ed51d675c6","Type":"ContainerStarted","Data":"ed56e576f1fd3818d01b8375026bb49b173aa68b846afd96b029977d54026496"} Jul 27 21:13:50.372101 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:50.372060 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" event={"ID":"53674d5c-bbff-4bd3-833f-f0ed51d675c6","Type":"ContainerStarted","Data":"7132ffdd85c60e365cd06370bf7afe1ed91860f3a717fef9465bb9b152245b04"} Jul 27 21:13:50.397809 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:50.397763 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" podStartSLOduration=1.397748907 podStartE2EDuration="1.397748907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:13:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:13:50.395568098 +0000 UTC m=+1358.918685848" watchObservedRunningTime="2026-07-27 21:13:50.397748907 +0000 UTC m=+1358.920866628" Jul 27 21:13:59.936145 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:59.936103 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:59.936145 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:59.936155 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:13:59.949132 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:13:59.949109 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:14:00.430180 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:00.430148 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:14:13.458060 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:13.458028 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf"] Jul 27 21:14:28.478296 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.478251 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" podUID="53674d5c-bbff-4bd3-833f-f0ed51d675c6" containerName="main" containerID="cri-o://ed56e576f1fd3818d01b8375026bb49b173aa68b846afd96b029977d54026496" gracePeriod=45 Jul 27 21:14:28.732588 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.732520 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:14:28.767904 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.767874 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-model-cache\") pod \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " Jul 27 21:14:28.768086 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.767927 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgcz9\" (UniqueName: \"kubernetes.io/projected/53674d5c-bbff-4bd3-833f-f0ed51d675c6-kube-api-access-xgcz9\") pod \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " Jul 27 21:14:28.768086 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.767957 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-dshm\") pod \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " Jul 27 21:14:28.768086 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.767987 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-home\") pod \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " Jul 27 21:14:28.768086 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.768038 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-tmp-dir\") pod \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " Jul 27 21:14:28.768086 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.768084 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53674d5c-bbff-4bd3-833f-f0ed51d675c6-tls-certs\") pod \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\" (UID: \"53674d5c-bbff-4bd3-833f-f0ed51d675c6\") " Jul 27 21:14:28.768385 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.768145 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-model-cache" (OuterVolumeSpecName: "model-cache") pod "53674d5c-bbff-4bd3-833f-f0ed51d675c6" (UID: "53674d5c-bbff-4bd3-833f-f0ed51d675c6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:14:28.768385 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.768318 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-home" (OuterVolumeSpecName: "home") pod "53674d5c-bbff-4bd3-833f-f0ed51d675c6" (UID: "53674d5c-bbff-4bd3-833f-f0ed51d675c6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:14:28.768385 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.768373 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-model-cache\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:14:28.768538 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.768390 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "53674d5c-bbff-4bd3-833f-f0ed51d675c6" (UID: "53674d5c-bbff-4bd3-833f-f0ed51d675c6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:14:28.770460 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.770431 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-dshm" (OuterVolumeSpecName: "dshm") pod "53674d5c-bbff-4bd3-833f-f0ed51d675c6" (UID: "53674d5c-bbff-4bd3-833f-f0ed51d675c6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:14:28.770572 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.770480 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53674d5c-bbff-4bd3-833f-f0ed51d675c6-kube-api-access-xgcz9" (OuterVolumeSpecName: "kube-api-access-xgcz9") pod "53674d5c-bbff-4bd3-833f-f0ed51d675c6" (UID: "53674d5c-bbff-4bd3-833f-f0ed51d675c6"). InnerVolumeSpecName "kube-api-access-xgcz9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:14:28.770572 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.770518 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53674d5c-bbff-4bd3-833f-f0ed51d675c6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "53674d5c-bbff-4bd3-833f-f0ed51d675c6" (UID: "53674d5c-bbff-4bd3-833f-f0ed51d675c6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 21:14:28.869298 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.869260 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-home\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:14:28.869298 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.869291 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-tmp-dir\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:14:28.869298 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.869302 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53674d5c-bbff-4bd3-833f-f0ed51d675c6-tls-certs\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:14:28.869520 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.869312 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgcz9\" (UniqueName: \"kubernetes.io/projected/53674d5c-bbff-4bd3-833f-f0ed51d675c6-kube-api-access-xgcz9\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:14:28.869520 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:28.869321 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53674d5c-bbff-4bd3-833f-f0ed51d675c6-dshm\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:14:29.563594 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:29.563557 2565 generic.go:356] "Generic (PLEG): container finished" podID="53674d5c-bbff-4bd3-833f-f0ed51d675c6" containerID="ed56e576f1fd3818d01b8375026bb49b173aa68b846afd96b029977d54026496" exitCode=0 Jul 27 21:14:29.564019 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:29.563603 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" Jul 27 21:14:29.564019 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:29.563652 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" event={"ID":"53674d5c-bbff-4bd3-833f-f0ed51d675c6","Type":"ContainerDied","Data":"ed56e576f1fd3818d01b8375026bb49b173aa68b846afd96b029977d54026496"} Jul 27 21:14:29.564019 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:29.563694 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf" event={"ID":"53674d5c-bbff-4bd3-833f-f0ed51d675c6","Type":"ContainerDied","Data":"7132ffdd85c60e365cd06370bf7afe1ed91860f3a717fef9465bb9b152245b04"} Jul 27 21:14:29.564019 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:29.563710 2565 scope.go:117] "RemoveContainer" containerID="ed56e576f1fd3818d01b8375026bb49b173aa68b846afd96b029977d54026496" Jul 27 21:14:29.573047 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:29.573028 2565 scope.go:117] "RemoveContainer" containerID="ed56e576f1fd3818d01b8375026bb49b173aa68b846afd96b029977d54026496" Jul 27 21:14:29.573335 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:14:29.573317 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed56e576f1fd3818d01b8375026bb49b173aa68b846afd96b029977d54026496\": container with ID starting with ed56e576f1fd3818d01b8375026bb49b173aa68b846afd96b029977d54026496 not found: ID does not exist" containerID="ed56e576f1fd3818d01b8375026bb49b173aa68b846afd96b029977d54026496" Jul 27 21:14:29.573409 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:29.573343 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed56e576f1fd3818d01b8375026bb49b173aa68b846afd96b029977d54026496"} err="failed to get container status \"ed56e576f1fd3818d01b8375026bb49b173aa68b846afd96b029977d54026496\": rpc error: code = NotFound desc = could not find container \"ed56e576f1fd3818d01b8375026bb49b173aa68b846afd96b029977d54026496\": container with ID starting with ed56e576f1fd3818d01b8375026bb49b173aa68b846afd96b029977d54026496 not found: ID does not exist" Jul 27 21:14:29.586329 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:29.586304 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf"] Jul 27 21:14:29.590576 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:29.590556 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-379c72fa-kserve-59cf5fd587-g8sqf"] Jul 27 21:14:30.052222 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:14:30.052191 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53674d5c-bbff-4bd3-833f-f0ed51d675c6" path="/var/lib/kubelet/pods/53674d5c-bbff-4bd3-833f-f0ed51d675c6/volumes" Jul 27 21:17:55.853091 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:55.853060 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7"] Jul 27 21:17:55.855450 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:55.853420 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53674d5c-bbff-4bd3-833f-f0ed51d675c6" containerName="main" Jul 27 21:17:55.855450 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:55.853431 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="53674d5c-bbff-4bd3-833f-f0ed51d675c6" containerName="main" Jul 27 21:17:55.855450 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:55.853487 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="53674d5c-bbff-4bd3-833f-f0ed51d675c6" containerName="main" Jul 27 21:17:55.856381 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:55.856365 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:55.859941 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:55.859922 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5f8eb8aa-openshift-default-dockercfg-d4qxl\"" Jul 27 21:17:55.891219 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:55.891194 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7"] Jul 27 21:17:55.901869 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:55.901847 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-workload-socket\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:55.901991 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:55.901885 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-podinfo\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:55.901991 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:55.901903 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae795bfe-43b2-4662-be67-a789a09b3aa8-istiod-ca-cert\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:55.901991 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:55.901976 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wdnn\" (UniqueName: \"kubernetes.io/projected/ae795bfe-43b2-4662-be67-a789a09b3aa8-kube-api-access-7wdnn\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:55.902160 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:55.902054 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-data\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:55.902160 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:55.902083 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-workload-certs\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:55.902283 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:55.902159 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-envoy\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:55.902283 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:55.902192 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-token\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:55.902283 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:55.902230 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-credential-socket\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.002750 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.002715 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-envoy\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.002750 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.002757 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-token\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.002998 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.002785 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-credential-socket\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.002998 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.002825 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-workload-socket\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.002998 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.002864 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-podinfo\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.002998 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.002885 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae795bfe-43b2-4662-be67-a789a09b3aa8-istiod-ca-cert\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.002998 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.002907 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wdnn\" (UniqueName: \"kubernetes.io/projected/ae795bfe-43b2-4662-be67-a789a09b3aa8-kube-api-access-7wdnn\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.002998 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.002954 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-data\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.002998 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.002977 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-workload-certs\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.003395 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.003276 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-credential-socket\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.003395 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.003291 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-workload-socket\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.003500 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.003413 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-workload-certs\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.003564 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.003525 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-data\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.003771 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.003746 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae795bfe-43b2-4662-be67-a789a09b3aa8-istiod-ca-cert\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.005221 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.005201 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-envoy\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.005503 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.005485 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-podinfo\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.011692 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.011668 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wdnn\" (UniqueName: \"kubernetes.io/projected/ae795bfe-43b2-4662-be67-a789a09b3aa8-kube-api-access-7wdnn\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.011804 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.011717 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-token\") pod \"e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.166162 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.166132 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:56.309838 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.309815 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7"] Jul 27 21:17:56.311656 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:17:56.311624 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae795bfe_43b2_4662_be67_a789a09b3aa8.slice/crio-7987f33b752f56aee8276456ca69dfcef702ef4501b4ededf35198b143ca6cf7 WatchSource:0}: Error finding container 7987f33b752f56aee8276456ca69dfcef702ef4501b4ededf35198b143ca6cf7: Status 404 returned error can't find the container with id 7987f33b752f56aee8276456ca69dfcef702ef4501b4ededf35198b143ca6cf7 Jul 27 21:17:56.314151 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.314119 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:17:56.314262 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.314188 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:17:56.314262 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.314220 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:17:56.572642 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.572548 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" event={"ID":"ae795bfe-43b2-4662-be67-a789a09b3aa8","Type":"ContainerStarted","Data":"dbbc5a83f08bd7a34c4787a386aa4e7cc767a6d2faaf402fb5fb93118b75a2d2"} Jul 27 21:17:56.572642 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.572625 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" event={"ID":"ae795bfe-43b2-4662-be67-a789a09b3aa8","Type":"ContainerStarted","Data":"7987f33b752f56aee8276456ca69dfcef702ef4501b4ededf35198b143ca6cf7"} Jul 27 21:17:56.593628 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:56.593585 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" podStartSLOduration=1.593571496 podStartE2EDuration="1.593571496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:17:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:17:56.591490116 +0000 UTC m=+1605.114607839" watchObservedRunningTime="2026-07-27 21:17:56.593571496 +0000 UTC m=+1605.116689217" Jul 27 21:17:57.166705 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:57.166664 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:57.171750 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:57.171725 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:57.577165 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:57.577075 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:17:57.578115 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:17:57.578097 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:18:09.639650 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:09.639616 2565 generic.go:356] "Generic (PLEG): container finished" podID="ae795bfe-43b2-4662-be67-a789a09b3aa8" containerID="dbbc5a83f08bd7a34c4787a386aa4e7cc767a6d2faaf402fb5fb93118b75a2d2" exitCode=0 Jul 27 21:18:09.640041 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:09.639688 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" event={"ID":"ae795bfe-43b2-4662-be67-a789a09b3aa8","Type":"ContainerDied","Data":"dbbc5a83f08bd7a34c4787a386aa4e7cc767a6d2faaf402fb5fb93118b75a2d2"} Jul 27 21:18:09.640082 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:09.640048 2565 scope.go:117] "RemoveContainer" containerID="dbbc5a83f08bd7a34c4787a386aa4e7cc767a6d2faaf402fb5fb93118b75a2d2" Jul 27 21:18:09.641009 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:09.640982 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:18:09.641064 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:09.641050 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:18:09.641099 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:09.641085 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:18:10.647676 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:10.647646 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" event={"ID":"ae795bfe-43b2-4662-be67-a789a09b3aa8","Type":"ContainerStarted","Data":"5e32dea5894f362443f8114cfba32937878bbc4f4297f4d5047e5c9e257e5e1b"} Jul 27 21:18:11.168221 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:11.168176 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:18:11.168528 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:11.168370 2565 patch_prober.go:28] interesting pod/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 27 21:18:11.168528 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:11.168430 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" podUID="ae795bfe-43b2-4662-be67-a789a09b3aa8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 27 21:18:12.167557 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:12.167518 2565 patch_prober.go:28] interesting pod/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 27 21:18:12.168102 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:12.168069 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" podUID="ae795bfe-43b2-4662-be67-a789a09b3aa8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 27 21:18:12.577390 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:12.577289 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:18:13.166816 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:13.166774 2565 patch_prober.go:28] interesting pod/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 27 21:18:13.167015 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:13.166850 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" podUID="ae795bfe-43b2-4662-be67-a789a09b3aa8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 27 21:18:14.167711 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:14.167673 2565 patch_prober.go:28] interesting pod/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 27 21:18:14.168332 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:14.168293 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" podUID="ae795bfe-43b2-4662-be67-a789a09b3aa8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 27 21:18:15.170922 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:15.170891 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:18:15.171840 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:15.171823 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:18:38.715660 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:38.715622 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7"] Jul 27 21:18:38.716184 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:38.715873 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" podUID="ae795bfe-43b2-4662-be67-a789a09b3aa8" containerName="istio-proxy" containerID="cri-o://5e32dea5894f362443f8114cfba32937878bbc4f4297f4d5047e5c9e257e5e1b" gracePeriod=30 Jul 27 21:18:39.781740 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.781706 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c"] Jul 27 21:18:39.785377 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.785352 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.788987 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.788965 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-20b824d1-gw-openshift-default-dockercfg-thss8\"" Jul 27 21:18:39.810596 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.810562 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c"] Jul 27 21:18:39.896939 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.896903 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-workload-certs\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.897111 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.896963 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/207d0a80-5600-47ea-b26c-35e46411d061-istio-podinfo\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.897111 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.897039 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/207d0a80-5600-47ea-b26c-35e46411d061-istiod-ca-cert\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.897111 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.897074 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-workload-socket\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.897111 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.897089 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-istio-data\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.897315 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.897126 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-istio-envoy\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.897315 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.897145 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-credential-socket\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.897315 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.897222 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvrct\" (UniqueName: \"kubernetes.io/projected/207d0a80-5600-47ea-b26c-35e46411d061-kube-api-access-cvrct\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.897315 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.897265 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/207d0a80-5600-47ea-b26c-35e46411d061-istio-token\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.997918 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.997887 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvrct\" (UniqueName: \"kubernetes.io/projected/207d0a80-5600-47ea-b26c-35e46411d061-kube-api-access-cvrct\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.997918 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.997922 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/207d0a80-5600-47ea-b26c-35e46411d061-istio-token\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.998146 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.997948 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-workload-certs\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.998146 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.997998 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/207d0a80-5600-47ea-b26c-35e46411d061-istio-podinfo\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.998146 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.998037 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/207d0a80-5600-47ea-b26c-35e46411d061-istiod-ca-cert\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.998146 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.998061 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-workload-socket\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.998146 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.998084 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-istio-data\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.998146 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.998128 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-istio-envoy\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.998483 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.998213 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-credential-socket\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.998483 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.998428 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-workload-certs\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.998585 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.998489 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-istio-data\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.998737 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.998703 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/207d0a80-5600-47ea-b26c-35e46411d061-istiod-ca-cert\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.998837 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.998780 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-workload-socket\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:39.999023 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:39.999002 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-credential-socket\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:40.000526 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:40.000500 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-istio-envoy\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:40.000726 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:40.000711 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/207d0a80-5600-47ea-b26c-35e46411d061-istio-podinfo\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:40.005827 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:40.005802 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/207d0a80-5600-47ea-b26c-35e46411d061-istio-token\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:40.006558 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:40.006536 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvrct\" (UniqueName: \"kubernetes.io/projected/207d0a80-5600-47ea-b26c-35e46411d061-kube-api-access-cvrct\") pod \"e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:40.096492 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:40.096408 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:40.228100 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:40.228071 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c"] Jul 27 21:18:40.229839 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:18:40.229810 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod207d0a80_5600_47ea_b26c_35e46411d061.slice/crio-2d852993590fca9eb7e2e93d162f9dd956129bbd2bcf03555d5bc7f4e46cfce0 WatchSource:0}: Error finding container 2d852993590fca9eb7e2e93d162f9dd956129bbd2bcf03555d5bc7f4e46cfce0: Status 404 returned error can't find the container with id 2d852993590fca9eb7e2e93d162f9dd956129bbd2bcf03555d5bc7f4e46cfce0 Jul 27 21:18:40.232040 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:40.232005 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:18:40.232121 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:40.232091 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:18:40.232152 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:40.232132 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:18:40.797443 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:40.797404 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" event={"ID":"207d0a80-5600-47ea-b26c-35e46411d061","Type":"ContainerStarted","Data":"33a162f41b0e0359bd7077aef45738fb5f577062474816f134a00d384b5f8ef4"} Jul 27 21:18:40.797916 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:40.797470 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" event={"ID":"207d0a80-5600-47ea-b26c-35e46411d061","Type":"ContainerStarted","Data":"2d852993590fca9eb7e2e93d162f9dd956129bbd2bcf03555d5bc7f4e46cfce0"} Jul 27 21:18:40.819608 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:40.819563 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" podStartSLOduration=1.8195481340000002 podStartE2EDuration="1.819548134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:18:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:18:40.818220011 +0000 UTC m=+1649.341337734" watchObservedRunningTime="2026-07-27 21:18:40.819548134 +0000 UTC m=+1649.342665855" Jul 27 21:18:41.097158 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:41.097077 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:41.101578 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:41.101554 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:41.801812 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:41.801776 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:41.802828 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:41.802807 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:43.976159 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:43.976100 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:18:44.036333 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.036302 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-workload-socket\") pod \"ae795bfe-43b2-4662-be67-a789a09b3aa8\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " Jul 27 21:18:44.036457 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.036357 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-token\") pod \"ae795bfe-43b2-4662-be67-a789a09b3aa8\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " Jul 27 21:18:44.036457 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.036379 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-workload-certs\") pod \"ae795bfe-43b2-4662-be67-a789a09b3aa8\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " Jul 27 21:18:44.036457 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.036411 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-credential-socket\") pod \"ae795bfe-43b2-4662-be67-a789a09b3aa8\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " Jul 27 21:18:44.036457 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.036439 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wdnn\" (UniqueName: \"kubernetes.io/projected/ae795bfe-43b2-4662-be67-a789a09b3aa8-kube-api-access-7wdnn\") pod \"ae795bfe-43b2-4662-be67-a789a09b3aa8\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " Jul 27 21:18:44.036658 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.036476 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-podinfo\") pod \"ae795bfe-43b2-4662-be67-a789a09b3aa8\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " Jul 27 21:18:44.036658 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.036518 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae795bfe-43b2-4662-be67-a789a09b3aa8-istiod-ca-cert\") pod \"ae795bfe-43b2-4662-be67-a789a09b3aa8\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " Jul 27 21:18:44.036658 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.036581 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ae795bfe-43b2-4662-be67-a789a09b3aa8" (UID: "ae795bfe-43b2-4662-be67-a789a09b3aa8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:18:44.036658 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.036645 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-data\") pod \"ae795bfe-43b2-4662-be67-a789a09b3aa8\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " Jul 27 21:18:44.036845 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.036683 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ae795bfe-43b2-4662-be67-a789a09b3aa8" (UID: "ae795bfe-43b2-4662-be67-a789a09b3aa8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:18:44.036845 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.036695 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ae795bfe-43b2-4662-be67-a789a09b3aa8" (UID: "ae795bfe-43b2-4662-be67-a789a09b3aa8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:18:44.036845 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.036695 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-envoy\") pod \"ae795bfe-43b2-4662-be67-a789a09b3aa8\" (UID: \"ae795bfe-43b2-4662-be67-a789a09b3aa8\") " Jul 27 21:18:44.036989 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.036955 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae795bfe-43b2-4662-be67-a789a09b3aa8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ae795bfe-43b2-4662-be67-a789a09b3aa8" (UID: "ae795bfe-43b2-4662-be67-a789a09b3aa8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 21:18:44.037040 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.037016 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-workload-certs\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:18:44.037091 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.037036 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-credential-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:18:44.037091 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.037051 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-workload-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:18:44.037538 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.037515 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-data" (OuterVolumeSpecName: "istio-data") pod "ae795bfe-43b2-4662-be67-a789a09b3aa8" (UID: "ae795bfe-43b2-4662-be67-a789a09b3aa8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:18:44.039414 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.039388 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-token" (OuterVolumeSpecName: "istio-token") pod "ae795bfe-43b2-4662-be67-a789a09b3aa8" (UID: "ae795bfe-43b2-4662-be67-a789a09b3aa8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:18:44.039505 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.039413 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae795bfe-43b2-4662-be67-a789a09b3aa8-kube-api-access-7wdnn" (OuterVolumeSpecName: "kube-api-access-7wdnn") pod "ae795bfe-43b2-4662-be67-a789a09b3aa8" (UID: "ae795bfe-43b2-4662-be67-a789a09b3aa8"). InnerVolumeSpecName "kube-api-access-7wdnn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:18:44.039505 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.039429 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ae795bfe-43b2-4662-be67-a789a09b3aa8" (UID: "ae795bfe-43b2-4662-be67-a789a09b3aa8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 27 21:18:44.039505 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.039461 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ae795bfe-43b2-4662-be67-a789a09b3aa8" (UID: "ae795bfe-43b2-4662-be67-a789a09b3aa8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:18:44.138391 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.138355 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-token\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:18:44.138391 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.138381 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wdnn\" (UniqueName: \"kubernetes.io/projected/ae795bfe-43b2-4662-be67-a789a09b3aa8-kube-api-access-7wdnn\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:18:44.138391 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.138391 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-podinfo\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:18:44.138391 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.138399 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae795bfe-43b2-4662-be67-a789a09b3aa8-istiod-ca-cert\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:18:44.138632 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.138407 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-data\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:18:44.138632 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.138414 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae795bfe-43b2-4662-be67-a789a09b3aa8-istio-envoy\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:18:44.818420 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.818384 2565 generic.go:356] "Generic (PLEG): container finished" podID="ae795bfe-43b2-4662-be67-a789a09b3aa8" containerID="5e32dea5894f362443f8114cfba32937878bbc4f4297f4d5047e5c9e257e5e1b" exitCode=0 Jul 27 21:18:44.818624 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.818428 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" Jul 27 21:18:44.818624 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.818474 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" event={"ID":"ae795bfe-43b2-4662-be67-a789a09b3aa8","Type":"ContainerDied","Data":"5e32dea5894f362443f8114cfba32937878bbc4f4297f4d5047e5c9e257e5e1b"} Jul 27 21:18:44.818624 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.818536 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7" event={"ID":"ae795bfe-43b2-4662-be67-a789a09b3aa8","Type":"ContainerDied","Data":"7987f33b752f56aee8276456ca69dfcef702ef4501b4ededf35198b143ca6cf7"} Jul 27 21:18:44.818624 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.818568 2565 scope.go:117] "RemoveContainer" containerID="5e32dea5894f362443f8114cfba32937878bbc4f4297f4d5047e5c9e257e5e1b" Jul 27 21:18:44.827459 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.827440 2565 scope.go:117] "RemoveContainer" containerID="dbbc5a83f08bd7a34c4787a386aa4e7cc767a6d2faaf402fb5fb93118b75a2d2" Jul 27 21:18:44.835879 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.835726 2565 scope.go:117] "RemoveContainer" containerID="5e32dea5894f362443f8114cfba32937878bbc4f4297f4d5047e5c9e257e5e1b" Jul 27 21:18:44.836181 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:18:44.836157 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e32dea5894f362443f8114cfba32937878bbc4f4297f4d5047e5c9e257e5e1b\": container with ID starting with 5e32dea5894f362443f8114cfba32937878bbc4f4297f4d5047e5c9e257e5e1b not found: ID does not exist" containerID="5e32dea5894f362443f8114cfba32937878bbc4f4297f4d5047e5c9e257e5e1b" Jul 27 21:18:44.836300 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.836187 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e32dea5894f362443f8114cfba32937878bbc4f4297f4d5047e5c9e257e5e1b"} err="failed to get container status \"5e32dea5894f362443f8114cfba32937878bbc4f4297f4d5047e5c9e257e5e1b\": rpc error: code = NotFound desc = could not find container \"5e32dea5894f362443f8114cfba32937878bbc4f4297f4d5047e5c9e257e5e1b\": container with ID starting with 5e32dea5894f362443f8114cfba32937878bbc4f4297f4d5047e5c9e257e5e1b not found: ID does not exist" Jul 27 21:18:44.836300 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.836205 2565 scope.go:117] "RemoveContainer" containerID="dbbc5a83f08bd7a34c4787a386aa4e7cc767a6d2faaf402fb5fb93118b75a2d2" Jul 27 21:18:44.836522 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:18:44.836500 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbbc5a83f08bd7a34c4787a386aa4e7cc767a6d2faaf402fb5fb93118b75a2d2\": container with ID starting with dbbc5a83f08bd7a34c4787a386aa4e7cc767a6d2faaf402fb5fb93118b75a2d2 not found: ID does not exist" containerID="dbbc5a83f08bd7a34c4787a386aa4e7cc767a6d2faaf402fb5fb93118b75a2d2" Jul 27 21:18:44.836586 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.836528 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbbc5a83f08bd7a34c4787a386aa4e7cc767a6d2faaf402fb5fb93118b75a2d2"} err="failed to get container status \"dbbc5a83f08bd7a34c4787a386aa4e7cc767a6d2faaf402fb5fb93118b75a2d2\": rpc error: code = NotFound desc = could not find container \"dbbc5a83f08bd7a34c4787a386aa4e7cc767a6d2faaf402fb5fb93118b75a2d2\": container with ID starting with dbbc5a83f08bd7a34c4787a386aa4e7cc767a6d2faaf402fb5fb93118b75a2d2 not found: ID does not exist" Jul 27 21:18:44.838522 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.838504 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7"] Jul 27 21:18:44.843974 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:44.843955 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5f8eb8aa-openshift-default-8677d7d44d-ft4t7"] Jul 27 21:18:46.052363 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:46.052335 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae795bfe-43b2-4662-be67-a789a09b3aa8" path="/var/lib/kubelet/pods/ae795bfe-43b2-4662-be67-a789a09b3aa8/volumes" Jul 27 21:18:56.803978 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:56.803940 2565 patch_prober.go:28] interesting pod/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 27 21:18:56.804406 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:56.804011 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" podUID="207d0a80-5600-47ea-b26c-35e46411d061" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 27 21:18:56.882416 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:56.882381 2565 generic.go:356] "Generic (PLEG): container finished" podID="207d0a80-5600-47ea-b26c-35e46411d061" containerID="33a162f41b0e0359bd7077aef45738fb5f577062474816f134a00d384b5f8ef4" exitCode=0 Jul 27 21:18:56.882597 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:56.882421 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" event={"ID":"207d0a80-5600-47ea-b26c-35e46411d061","Type":"ContainerDied","Data":"33a162f41b0e0359bd7077aef45738fb5f577062474816f134a00d384b5f8ef4"} Jul 27 21:18:56.882865 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:56.882845 2565 scope.go:117] "RemoveContainer" containerID="33a162f41b0e0359bd7077aef45738fb5f577062474816f134a00d384b5f8ef4" Jul 27 21:18:56.883909 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:56.883879 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:18:56.883973 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:56.883958 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:18:56.884006 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:56.883995 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:18:57.893893 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:57.893859 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" event={"ID":"207d0a80-5600-47ea-b26c-35e46411d061","Type":"ContainerStarted","Data":"6b81e6017f2f8f0ff1f91ed31d55fc5ed54feb8e6a83f26246ead547ca4ce115"} Jul 27 21:18:58.097013 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:58.096972 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:18:58.097290 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:58.097268 2565 patch_prober.go:28] interesting pod/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 27 21:18:58.097402 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:58.097328 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" podUID="207d0a80-5600-47ea-b26c-35e46411d061" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 27 21:18:59.097437 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:59.097397 2565 patch_prober.go:28] interesting pod/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 27 21:18:59.097837 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:18:59.097455 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" podUID="207d0a80-5600-47ea-b26c-35e46411d061" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 27 21:19:00.097400 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:00.097367 2565 patch_prober.go:28] interesting pod/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 27 21:19:00.097586 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:00.097424 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" podUID="207d0a80-5600-47ea-b26c-35e46411d061" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 27 21:19:01.096959 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:01.096920 2565 patch_prober.go:28] interesting pod/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 27 21:19:01.097139 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:01.096981 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" podUID="207d0a80-5600-47ea-b26c-35e46411d061" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 27 21:19:02.100982 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:02.100949 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:19:02.101376 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:02.101255 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:19:02.101906 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:02.101890 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:19:22.474879 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:22.474848 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c"] Jul 27 21:19:22.475404 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:22.475082 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" podUID="207d0a80-5600-47ea-b26c-35e46411d061" containerName="istio-proxy" containerID="cri-o://6b81e6017f2f8f0ff1f91ed31d55fc5ed54feb8e6a83f26246ead547ca4ce115" gracePeriod=30 Jul 27 21:19:23.226010 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.225979 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff"] Jul 27 21:19:23.226338 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.226326 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae795bfe-43b2-4662-be67-a789a09b3aa8" containerName="istio-proxy" Jul 27 21:19:23.226338 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.226339 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae795bfe-43b2-4662-be67-a789a09b3aa8" containerName="istio-proxy" Jul 27 21:19:23.226338 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.226353 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae795bfe-43b2-4662-be67-a789a09b3aa8" containerName="istio-proxy" Jul 27 21:19:23.226338 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.226359 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae795bfe-43b2-4662-be67-a789a09b3aa8" containerName="istio-proxy" Jul 27 21:19:23.226536 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.226427 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae795bfe-43b2-4662-be67-a789a09b3aa8" containerName="istio-proxy" Jul 27 21:19:23.226536 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.226439 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae795bfe-43b2-4662-be67-a789a09b3aa8" containerName="istio-proxy" Jul 27 21:19:23.229491 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.229445 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.232407 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.232385 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-fc269dcd-openshift-default-dockercfg-6zqmv\"" Jul 27 21:19:23.243665 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.243643 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff"] Jul 27 21:19:23.398388 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.398353 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-workload-certs\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.398388 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.398394 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn96s\" (UniqueName: \"kubernetes.io/projected/bf226b23-266c-4e59-bc7e-e8fb019bbf52-kube-api-access-kn96s\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.398611 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.398421 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-data\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.398611 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.398471 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-podinfo\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.398611 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.398495 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istiod-ca-cert\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.398611 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.398551 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-workload-socket\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.398611 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.398570 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-credential-socket\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.398807 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.398617 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-token\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.398807 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.398670 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-envoy\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.499806 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.499717 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-envoy\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.499806 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.499766 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-workload-certs\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.499806 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.499785 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn96s\" (UniqueName: \"kubernetes.io/projected/bf226b23-266c-4e59-bc7e-e8fb019bbf52-kube-api-access-kn96s\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.499806 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.499808 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-data\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.500461 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.499832 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-podinfo\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.500461 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.499857 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istiod-ca-cert\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.500461 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.499949 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-workload-socket\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.500461 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.499996 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-credential-socket\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.500461 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.500028 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-token\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.500461 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.500216 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-workload-certs\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.500461 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.500281 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-data\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.500733 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.500474 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-credential-socket\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.500733 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.500551 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-workload-socket\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.500856 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.500835 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istiod-ca-cert\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.502198 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.502176 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-envoy\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.502459 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.502441 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-podinfo\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.508524 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.508505 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-token\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.508703 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.508687 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn96s\" (UniqueName: \"kubernetes.io/projected/bf226b23-266c-4e59-bc7e-e8fb019bbf52-kube-api-access-kn96s\") pod \"e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.540107 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.540077 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:23.680007 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.679985 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff"] Jul 27 21:19:23.682267 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:19:23.682216 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf226b23_266c_4e59_bc7e_e8fb019bbf52.slice/crio-92b325b4476d17d9e46b462538a6d98422711bf688277556258b6ed5740894d6 WatchSource:0}: Error finding container 92b325b4476d17d9e46b462538a6d98422711bf688277556258b6ed5740894d6: Status 404 returned error can't find the container with id 92b325b4476d17d9e46b462538a6d98422711bf688277556258b6ed5740894d6 Jul 27 21:19:23.684697 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.684667 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:19:23.684767 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.684730 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:19:23.684767 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:23.684758 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:19:24.020818 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:24.020785 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" event={"ID":"bf226b23-266c-4e59-bc7e-e8fb019bbf52","Type":"ContainerStarted","Data":"cffcc3c6574e4c4fbc3b61c2e4fee514c84ed4a090b328aecb6fad97a1d9dcda"} Jul 27 21:19:24.021007 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:24.020831 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" event={"ID":"bf226b23-266c-4e59-bc7e-e8fb019bbf52","Type":"ContainerStarted","Data":"92b325b4476d17d9e46b462538a6d98422711bf688277556258b6ed5740894d6"} Jul 27 21:19:24.041476 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:24.041425 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" podStartSLOduration=1.041408835 podStartE2EDuration="1.041408835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:19:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:19:24.039723669 +0000 UTC m=+1692.562841393" watchObservedRunningTime="2026-07-27 21:19:24.041408835 +0000 UTC m=+1692.564526558" Jul 27 21:19:24.540467 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:24.540426 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:24.545100 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:24.545074 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:25.026259 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:25.026205 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:25.027118 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:25.027100 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:27.733635 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.733609 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:19:27.737828 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.737770 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-workload-certs\") pod \"207d0a80-5600-47ea-b26c-35e46411d061\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " Jul 27 21:19:27.737941 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.737881 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-istio-envoy\") pod \"207d0a80-5600-47ea-b26c-35e46411d061\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " Jul 27 21:19:27.737941 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.737929 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-credential-socket\") pod \"207d0a80-5600-47ea-b26c-35e46411d061\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " Jul 27 21:19:27.738061 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.737951 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvrct\" (UniqueName: \"kubernetes.io/projected/207d0a80-5600-47ea-b26c-35e46411d061-kube-api-access-cvrct\") pod \"207d0a80-5600-47ea-b26c-35e46411d061\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " Jul 27 21:19:27.738061 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.737985 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-workload-socket\") pod \"207d0a80-5600-47ea-b26c-35e46411d061\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " Jul 27 21:19:27.738061 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.738051 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-istio-data\") pod \"207d0a80-5600-47ea-b26c-35e46411d061\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " Jul 27 21:19:27.738208 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.738085 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/207d0a80-5600-47ea-b26c-35e46411d061-istio-podinfo\") pod \"207d0a80-5600-47ea-b26c-35e46411d061\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " Jul 27 21:19:27.738208 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.738128 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/207d0a80-5600-47ea-b26c-35e46411d061-istiod-ca-cert\") pod \"207d0a80-5600-47ea-b26c-35e46411d061\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " Jul 27 21:19:27.738208 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.738155 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/207d0a80-5600-47ea-b26c-35e46411d061-istio-token\") pod \"207d0a80-5600-47ea-b26c-35e46411d061\" (UID: \"207d0a80-5600-47ea-b26c-35e46411d061\") " Jul 27 21:19:27.738386 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.737984 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "207d0a80-5600-47ea-b26c-35e46411d061" (UID: "207d0a80-5600-47ea-b26c-35e46411d061"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:19:27.738440 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.738217 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "207d0a80-5600-47ea-b26c-35e46411d061" (UID: "207d0a80-5600-47ea-b26c-35e46411d061"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:19:27.738493 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.738463 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "207d0a80-5600-47ea-b26c-35e46411d061" (UID: "207d0a80-5600-47ea-b26c-35e46411d061"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:19:27.738493 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.738488 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-workload-certs\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:19:27.738596 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.738506 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-credential-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:19:27.738745 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.738727 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/207d0a80-5600-47ea-b26c-35e46411d061-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "207d0a80-5600-47ea-b26c-35e46411d061" (UID: "207d0a80-5600-47ea-b26c-35e46411d061"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 21:19:27.739380 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.739354 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-istio-data" (OuterVolumeSpecName: "istio-data") pod "207d0a80-5600-47ea-b26c-35e46411d061" (UID: "207d0a80-5600-47ea-b26c-35e46411d061"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:19:27.740279 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.740260 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "207d0a80-5600-47ea-b26c-35e46411d061" (UID: "207d0a80-5600-47ea-b26c-35e46411d061"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:19:27.740593 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.740575 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/207d0a80-5600-47ea-b26c-35e46411d061-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "207d0a80-5600-47ea-b26c-35e46411d061" (UID: "207d0a80-5600-47ea-b26c-35e46411d061"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 27 21:19:27.740803 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.740787 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/207d0a80-5600-47ea-b26c-35e46411d061-kube-api-access-cvrct" (OuterVolumeSpecName: "kube-api-access-cvrct") pod "207d0a80-5600-47ea-b26c-35e46411d061" (UID: "207d0a80-5600-47ea-b26c-35e46411d061"). InnerVolumeSpecName "kube-api-access-cvrct". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:19:27.740930 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.740908 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/207d0a80-5600-47ea-b26c-35e46411d061-istio-token" (OuterVolumeSpecName: "istio-token") pod "207d0a80-5600-47ea-b26c-35e46411d061" (UID: "207d0a80-5600-47ea-b26c-35e46411d061"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:19:27.839752 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.839708 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/207d0a80-5600-47ea-b26c-35e46411d061-istiod-ca-cert\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:19:27.839752 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.839741 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/207d0a80-5600-47ea-b26c-35e46411d061-istio-token\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:19:27.839752 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.839749 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-istio-envoy\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:19:27.839752 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.839757 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvrct\" (UniqueName: \"kubernetes.io/projected/207d0a80-5600-47ea-b26c-35e46411d061-kube-api-access-cvrct\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:19:27.840094 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.839767 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-workload-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:19:27.840094 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.839775 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/207d0a80-5600-47ea-b26c-35e46411d061-istio-data\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:19:27.840094 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:27.839783 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/207d0a80-5600-47ea-b26c-35e46411d061-istio-podinfo\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:19:28.043281 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:28.043164 2565 generic.go:356] "Generic (PLEG): container finished" podID="207d0a80-5600-47ea-b26c-35e46411d061" containerID="6b81e6017f2f8f0ff1f91ed31d55fc5ed54feb8e6a83f26246ead547ca4ce115" exitCode=0 Jul 27 21:19:28.043281 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:28.043208 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" event={"ID":"207d0a80-5600-47ea-b26c-35e46411d061","Type":"ContainerDied","Data":"6b81e6017f2f8f0ff1f91ed31d55fc5ed54feb8e6a83f26246ead547ca4ce115"} Jul 27 21:19:28.043281 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:28.043221 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" Jul 27 21:19:28.043496 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:28.043283 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c" event={"ID":"207d0a80-5600-47ea-b26c-35e46411d061","Type":"ContainerDied","Data":"2d852993590fca9eb7e2e93d162f9dd956129bbd2bcf03555d5bc7f4e46cfce0"} Jul 27 21:19:28.043496 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:28.043309 2565 scope.go:117] "RemoveContainer" containerID="6b81e6017f2f8f0ff1f91ed31d55fc5ed54feb8e6a83f26246ead547ca4ce115" Jul 27 21:19:28.059880 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:28.059857 2565 scope.go:117] "RemoveContainer" containerID="33a162f41b0e0359bd7077aef45738fb5f577062474816f134a00d384b5f8ef4" Jul 27 21:19:28.068571 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:28.068552 2565 scope.go:117] "RemoveContainer" containerID="6b81e6017f2f8f0ff1f91ed31d55fc5ed54feb8e6a83f26246ead547ca4ce115" Jul 27 21:19:28.068851 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:19:28.068831 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b81e6017f2f8f0ff1f91ed31d55fc5ed54feb8e6a83f26246ead547ca4ce115\": container with ID starting with 6b81e6017f2f8f0ff1f91ed31d55fc5ed54feb8e6a83f26246ead547ca4ce115 not found: ID does not exist" containerID="6b81e6017f2f8f0ff1f91ed31d55fc5ed54feb8e6a83f26246ead547ca4ce115" Jul 27 21:19:28.068923 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:28.068862 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b81e6017f2f8f0ff1f91ed31d55fc5ed54feb8e6a83f26246ead547ca4ce115"} err="failed to get container status \"6b81e6017f2f8f0ff1f91ed31d55fc5ed54feb8e6a83f26246ead547ca4ce115\": rpc error: code = NotFound desc = could not find container \"6b81e6017f2f8f0ff1f91ed31d55fc5ed54feb8e6a83f26246ead547ca4ce115\": container with ID starting with 6b81e6017f2f8f0ff1f91ed31d55fc5ed54feb8e6a83f26246ead547ca4ce115 not found: ID does not exist" Jul 27 21:19:28.068923 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:28.068883 2565 scope.go:117] "RemoveContainer" containerID="33a162f41b0e0359bd7077aef45738fb5f577062474816f134a00d384b5f8ef4" Jul 27 21:19:28.069114 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:19:28.069098 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33a162f41b0e0359bd7077aef45738fb5f577062474816f134a00d384b5f8ef4\": container with ID starting with 33a162f41b0e0359bd7077aef45738fb5f577062474816f134a00d384b5f8ef4 not found: ID does not exist" containerID="33a162f41b0e0359bd7077aef45738fb5f577062474816f134a00d384b5f8ef4" Jul 27 21:19:28.069192 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:28.069119 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33a162f41b0e0359bd7077aef45738fb5f577062474816f134a00d384b5f8ef4"} err="failed to get container status \"33a162f41b0e0359bd7077aef45738fb5f577062474816f134a00d384b5f8ef4\": rpc error: code = NotFound desc = could not find container \"33a162f41b0e0359bd7077aef45738fb5f577062474816f134a00d384b5f8ef4\": container with ID starting with 33a162f41b0e0359bd7077aef45738fb5f577062474816f134a00d384b5f8ef4 not found: ID does not exist" Jul 27 21:19:28.070567 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:28.070546 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c"] Jul 27 21:19:28.075982 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:28.075960 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-20b824d1-gw-openshift-default-7fb98fdc64-dft8c"] Jul 27 21:19:30.053769 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:30.053731 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="207d0a80-5600-47ea-b26c-35e46411d061" path="/var/lib/kubelet/pods/207d0a80-5600-47ea-b26c-35e46411d061/volumes" Jul 27 21:19:40.027962 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:40.027859 2565 patch_prober.go:28] interesting pod/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 27 21:19:40.027962 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:40.027913 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" podUID="bf226b23-266c-4e59-bc7e-e8fb019bbf52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 27 21:19:40.119222 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:40.119190 2565 generic.go:356] "Generic (PLEG): container finished" podID="bf226b23-266c-4e59-bc7e-e8fb019bbf52" containerID="cffcc3c6574e4c4fbc3b61c2e4fee514c84ed4a090b328aecb6fad97a1d9dcda" exitCode=0 Jul 27 21:19:40.119406 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:40.119256 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" event={"ID":"bf226b23-266c-4e59-bc7e-e8fb019bbf52","Type":"ContainerDied","Data":"cffcc3c6574e4c4fbc3b61c2e4fee514c84ed4a090b328aecb6fad97a1d9dcda"} Jul 27 21:19:40.119615 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:40.119602 2565 scope.go:117] "RemoveContainer" containerID="cffcc3c6574e4c4fbc3b61c2e4fee514c84ed4a090b328aecb6fad97a1d9dcda" Jul 27 21:19:40.120663 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:40.120633 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:19:40.120717 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:40.120706 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:19:40.120755 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:40.120742 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:19:41.126253 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:41.126196 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" event={"ID":"bf226b23-266c-4e59-bc7e-e8fb019bbf52","Type":"ContainerStarted","Data":"08bfccca72906e74c793c64d9ad06e8948117340d3c4017cf61c5a7911cf9994"} Jul 27 21:19:41.540580 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:41.540546 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:41.540942 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:41.540919 2565 patch_prober.go:28] interesting pod/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 27 21:19:41.541003 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:41.540982 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" podUID="bf226b23-266c-4e59-bc7e-e8fb019bbf52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 27 21:19:42.541288 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:42.541251 2565 patch_prober.go:28] interesting pod/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 27 21:19:42.541718 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:42.541326 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" podUID="bf226b23-266c-4e59-bc7e-e8fb019bbf52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 27 21:19:43.540757 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:43.540721 2565 patch_prober.go:28] interesting pod/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 27 21:19:43.540934 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:43.540774 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" podUID="bf226b23-266c-4e59-bc7e-e8fb019bbf52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 27 21:19:44.541056 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:44.541021 2565 patch_prober.go:28] interesting pod/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 27 21:19:44.541507 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:44.541084 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" podUID="bf226b23-266c-4e59-bc7e-e8fb019bbf52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 27 21:19:45.544410 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:45.544371 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:45.544795 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:45.544714 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:19:45.545324 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:19:45.545306 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:20:07.286677 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:07.286643 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff"] Jul 27 21:20:07.287168 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:07.286863 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" podUID="bf226b23-266c-4e59-bc7e-e8fb019bbf52" containerName="istio-proxy" containerID="cri-o://08bfccca72906e74c793c64d9ad06e8948117340d3c4017cf61c5a7911cf9994" gracePeriod=30 Jul 27 21:20:08.034457 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.034424 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq"] Jul 27 21:20:08.034785 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.034773 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="207d0a80-5600-47ea-b26c-35e46411d061" containerName="istio-proxy" Jul 27 21:20:08.034836 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.034787 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="207d0a80-5600-47ea-b26c-35e46411d061" containerName="istio-proxy" Jul 27 21:20:08.034836 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.034799 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="207d0a80-5600-47ea-b26c-35e46411d061" containerName="istio-proxy" Jul 27 21:20:08.034836 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.034805 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="207d0a80-5600-47ea-b26c-35e46411d061" containerName="istio-proxy" Jul 27 21:20:08.034993 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.034873 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="207d0a80-5600-47ea-b26c-35e46411d061" containerName="istio-proxy" Jul 27 21:20:08.034993 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.034881 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="207d0a80-5600-47ea-b26c-35e46411d061" containerName="istio-proxy" Jul 27 21:20:08.037807 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.037792 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.043549 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.043529 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-a9503936-openshift-default-dockercfg-krd9w\"" Jul 27 21:20:08.080333 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.080309 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq"] Jul 27 21:20:08.198653 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.198615 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-workload-socket\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.198814 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.198661 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-credential-socket\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.198814 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.198691 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-workload-certs\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.198814 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.198722 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-envoy\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.198814 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.198744 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-podinfo\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.198814 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.198784 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istiod-ca-cert\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.198980 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.198856 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ks4s\" (UniqueName: \"kubernetes.io/projected/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-kube-api-access-2ks4s\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.198980 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.198893 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-data\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.198980 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.198917 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-token\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.299920 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.299831 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-workload-socket\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.299920 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.299875 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-credential-socket\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.299920 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.299895 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-workload-certs\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.299920 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.299916 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-envoy\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.300547 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.299942 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-podinfo\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.300547 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.299990 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istiod-ca-cert\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.300547 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.300048 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ks4s\" (UniqueName: \"kubernetes.io/projected/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-kube-api-access-2ks4s\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.300547 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.300083 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-data\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.300547 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.300109 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-token\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.300547 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.300320 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-credential-socket\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.300547 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.300398 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-workload-certs\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.300825 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.300572 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-workload-socket\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.300825 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.300578 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-data\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.300825 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.300739 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istiod-ca-cert\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.302970 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.302947 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-envoy\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.303124 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.303109 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-podinfo\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.316932 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.316904 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ks4s\" (UniqueName: \"kubernetes.io/projected/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-kube-api-access-2ks4s\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.327848 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.327816 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-token\") pod \"e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.347557 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.347525 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:08.512954 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.512928 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq"] Jul 27 21:20:08.514388 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:20:08.514356 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75ee05aa_3ea9_448b_9bb2_296d68a42d2e.slice/crio-a618769ff839eb4a8d0ac017d857274f5febbb75e9986a1b7e70f5528f693d54 WatchSource:0}: Error finding container a618769ff839eb4a8d0ac017d857274f5febbb75e9986a1b7e70f5528f693d54: Status 404 returned error can't find the container with id a618769ff839eb4a8d0ac017d857274f5febbb75e9986a1b7e70f5528f693d54 Jul 27 21:20:08.516400 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.516365 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:20:08.516493 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.516433 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:20:08.516493 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:08.516465 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:20:09.265588 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:09.265537 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" event={"ID":"75ee05aa-3ea9-448b-9bb2-296d68a42d2e","Type":"ContainerStarted","Data":"9f8fafab4b06a339852710bc07c67f43104b63297738963d390d7f73e113124e"} Jul 27 21:20:09.265779 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:09.265609 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" event={"ID":"75ee05aa-3ea9-448b-9bb2-296d68a42d2e","Type":"ContainerStarted","Data":"a618769ff839eb4a8d0ac017d857274f5febbb75e9986a1b7e70f5528f693d54"} Jul 27 21:20:09.286443 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:09.286382 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" podStartSLOduration=1.286361645 podStartE2EDuration="1.286361645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:20:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:20:09.28559957 +0000 UTC m=+1737.808717309" watchObservedRunningTime="2026-07-27 21:20:09.286361645 +0000 UTC m=+1737.809479367" Jul 27 21:20:09.347817 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:09.347779 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:09.352444 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:09.352416 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:10.270441 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:10.270414 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:10.271498 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:10.271476 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:12.533767 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.533740 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:20:12.644677 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.644598 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-envoy\") pod \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " Jul 27 21:20:12.644677 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.644641 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-workload-socket\") pod \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " Jul 27 21:20:12.644677 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.644666 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istiod-ca-cert\") pod \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " Jul 27 21:20:12.644952 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.644716 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-podinfo\") pod \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " Jul 27 21:20:12.644952 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.644769 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kn96s\" (UniqueName: \"kubernetes.io/projected/bf226b23-266c-4e59-bc7e-e8fb019bbf52-kube-api-access-kn96s\") pod \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " Jul 27 21:20:12.644952 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.644887 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-token\") pod \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " Jul 27 21:20:12.644952 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.644928 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bf226b23-266c-4e59-bc7e-e8fb019bbf52" (UID: "bf226b23-266c-4e59-bc7e-e8fb019bbf52"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:20:12.645260 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.644954 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-workload-certs\") pod \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " Jul 27 21:20:12.645260 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.644983 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-credential-socket\") pod \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " Jul 27 21:20:12.645260 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.645010 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-data\") pod \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\" (UID: \"bf226b23-266c-4e59-bc7e-e8fb019bbf52\") " Jul 27 21:20:12.645260 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.645176 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bf226b23-266c-4e59-bc7e-e8fb019bbf52" (UID: "bf226b23-266c-4e59-bc7e-e8fb019bbf52"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 21:20:12.645457 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.645406 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-workload-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:20:12.645457 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.645427 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istiod-ca-cert\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:20:12.645457 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.645432 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bf226b23-266c-4e59-bc7e-e8fb019bbf52" (UID: "bf226b23-266c-4e59-bc7e-e8fb019bbf52"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:20:12.645662 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.645642 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bf226b23-266c-4e59-bc7e-e8fb019bbf52" (UID: "bf226b23-266c-4e59-bc7e-e8fb019bbf52"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:20:12.646121 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.646099 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-data" (OuterVolumeSpecName: "istio-data") pod "bf226b23-266c-4e59-bc7e-e8fb019bbf52" (UID: "bf226b23-266c-4e59-bc7e-e8fb019bbf52"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:20:12.647032 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.647007 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bf226b23-266c-4e59-bc7e-e8fb019bbf52" (UID: "bf226b23-266c-4e59-bc7e-e8fb019bbf52"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:20:12.647119 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.647078 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bf226b23-266c-4e59-bc7e-e8fb019bbf52" (UID: "bf226b23-266c-4e59-bc7e-e8fb019bbf52"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 27 21:20:12.647601 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.647580 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf226b23-266c-4e59-bc7e-e8fb019bbf52-kube-api-access-kn96s" (OuterVolumeSpecName: "kube-api-access-kn96s") pod "bf226b23-266c-4e59-bc7e-e8fb019bbf52" (UID: "bf226b23-266c-4e59-bc7e-e8fb019bbf52"). InnerVolumeSpecName "kube-api-access-kn96s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:20:12.647666 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.647594 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-token" (OuterVolumeSpecName: "istio-token") pod "bf226b23-266c-4e59-bc7e-e8fb019bbf52" (UID: "bf226b23-266c-4e59-bc7e-e8fb019bbf52"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:20:12.746812 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.746780 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-podinfo\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:20:12.746812 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.746807 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kn96s\" (UniqueName: \"kubernetes.io/projected/bf226b23-266c-4e59-bc7e-e8fb019bbf52-kube-api-access-kn96s\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:20:12.746812 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.746816 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-token\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:20:12.746812 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.746824 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-workload-certs\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:20:12.747081 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.746833 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-credential-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:20:12.747081 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.746841 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-data\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:20:12.747081 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:12.746849 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf226b23-266c-4e59-bc7e-e8fb019bbf52-istio-envoy\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:20:13.286176 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:13.286143 2565 generic.go:356] "Generic (PLEG): container finished" podID="bf226b23-266c-4e59-bc7e-e8fb019bbf52" containerID="08bfccca72906e74c793c64d9ad06e8948117340d3c4017cf61c5a7911cf9994" exitCode=0 Jul 27 21:20:13.286370 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:13.286187 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" event={"ID":"bf226b23-266c-4e59-bc7e-e8fb019bbf52","Type":"ContainerDied","Data":"08bfccca72906e74c793c64d9ad06e8948117340d3c4017cf61c5a7911cf9994"} Jul 27 21:20:13.286370 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:13.286194 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" Jul 27 21:20:13.286370 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:13.286246 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff" event={"ID":"bf226b23-266c-4e59-bc7e-e8fb019bbf52","Type":"ContainerDied","Data":"92b325b4476d17d9e46b462538a6d98422711bf688277556258b6ed5740894d6"} Jul 27 21:20:13.286370 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:13.286268 2565 scope.go:117] "RemoveContainer" containerID="08bfccca72906e74c793c64d9ad06e8948117340d3c4017cf61c5a7911cf9994" Jul 27 21:20:13.295599 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:13.295579 2565 scope.go:117] "RemoveContainer" containerID="cffcc3c6574e4c4fbc3b61c2e4fee514c84ed4a090b328aecb6fad97a1d9dcda" Jul 27 21:20:13.303207 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:13.303191 2565 scope.go:117] "RemoveContainer" containerID="08bfccca72906e74c793c64d9ad06e8948117340d3c4017cf61c5a7911cf9994" Jul 27 21:20:13.303486 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:20:13.303468 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08bfccca72906e74c793c64d9ad06e8948117340d3c4017cf61c5a7911cf9994\": container with ID starting with 08bfccca72906e74c793c64d9ad06e8948117340d3c4017cf61c5a7911cf9994 not found: ID does not exist" containerID="08bfccca72906e74c793c64d9ad06e8948117340d3c4017cf61c5a7911cf9994" Jul 27 21:20:13.303543 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:13.303495 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08bfccca72906e74c793c64d9ad06e8948117340d3c4017cf61c5a7911cf9994"} err="failed to get container status \"08bfccca72906e74c793c64d9ad06e8948117340d3c4017cf61c5a7911cf9994\": rpc error: code = NotFound desc = could not find container \"08bfccca72906e74c793c64d9ad06e8948117340d3c4017cf61c5a7911cf9994\": container with ID starting with 08bfccca72906e74c793c64d9ad06e8948117340d3c4017cf61c5a7911cf9994 not found: ID does not exist" Jul 27 21:20:13.303543 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:13.303511 2565 scope.go:117] "RemoveContainer" containerID="cffcc3c6574e4c4fbc3b61c2e4fee514c84ed4a090b328aecb6fad97a1d9dcda" Jul 27 21:20:13.303745 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:20:13.303728 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cffcc3c6574e4c4fbc3b61c2e4fee514c84ed4a090b328aecb6fad97a1d9dcda\": container with ID starting with cffcc3c6574e4c4fbc3b61c2e4fee514c84ed4a090b328aecb6fad97a1d9dcda not found: ID does not exist" containerID="cffcc3c6574e4c4fbc3b61c2e4fee514c84ed4a090b328aecb6fad97a1d9dcda" Jul 27 21:20:13.303781 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:13.303749 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cffcc3c6574e4c4fbc3b61c2e4fee514c84ed4a090b328aecb6fad97a1d9dcda"} err="failed to get container status \"cffcc3c6574e4c4fbc3b61c2e4fee514c84ed4a090b328aecb6fad97a1d9dcda\": rpc error: code = NotFound desc = could not find container \"cffcc3c6574e4c4fbc3b61c2e4fee514c84ed4a090b328aecb6fad97a1d9dcda\": container with ID starting with cffcc3c6574e4c4fbc3b61c2e4fee514c84ed4a090b328aecb6fad97a1d9dcda not found: ID does not exist" Jul 27 21:20:13.307920 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:13.307896 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff"] Jul 27 21:20:13.309826 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:13.309808 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-fc269dcd-openshift-default-bf5c749d-55fff"] Jul 27 21:20:14.052769 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:14.052737 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf226b23-266c-4e59-bc7e-e8fb019bbf52" path="/var/lib/kubelet/pods/bf226b23-266c-4e59-bc7e-e8fb019bbf52/volumes" Jul 27 21:20:24.344856 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:24.344824 2565 generic.go:356] "Generic (PLEG): container finished" podID="75ee05aa-3ea9-448b-9bb2-296d68a42d2e" containerID="9f8fafab4b06a339852710bc07c67f43104b63297738963d390d7f73e113124e" exitCode=0 Jul 27 21:20:24.345317 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:24.344873 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" event={"ID":"75ee05aa-3ea9-448b-9bb2-296d68a42d2e","Type":"ContainerDied","Data":"9f8fafab4b06a339852710bc07c67f43104b63297738963d390d7f73e113124e"} Jul 27 21:20:24.345317 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:24.345290 2565 scope.go:117] "RemoveContainer" containerID="9f8fafab4b06a339852710bc07c67f43104b63297738963d390d7f73e113124e" Jul 27 21:20:24.346301 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:24.346270 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:20:24.346363 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:24.346341 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:20:24.346412 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:24.346375 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:20:25.270869 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:25.270816 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:25.354796 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:25.354756 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" event={"ID":"75ee05aa-3ea9-448b-9bb2-296d68a42d2e","Type":"ContainerStarted","Data":"fb48c4dec743895d93a3da2b7047216829e51fd9b8eadebe5ddd6731226d2ba1"} Jul 27 21:20:26.348616 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:26.348576 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:26.349765 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:26.349746 2565 patch_prober.go:28] interesting pod/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 27 21:20:26.349825 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:26.349790 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" podUID="75ee05aa-3ea9-448b-9bb2-296d68a42d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 27 21:20:27.348043 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:27.348005 2565 patch_prober.go:28] interesting pod/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 27 21:20:27.348435 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:27.348061 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" podUID="75ee05aa-3ea9-448b-9bb2-296d68a42d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 27 21:20:28.348569 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:28.348531 2565 patch_prober.go:28] interesting pod/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 27 21:20:28.348968 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:28.348589 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" podUID="75ee05aa-3ea9-448b-9bb2-296d68a42d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 27 21:20:29.348126 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:29.348092 2565 patch_prober.go:28] interesting pod/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 27 21:20:29.348329 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:29.348150 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" podUID="75ee05aa-3ea9-448b-9bb2-296d68a42d2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 27 21:20:30.352313 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:30.352285 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:30.352700 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:30.352553 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:30.353103 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:30.353087 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:50.707585 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:50.707545 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq"] Jul 27 21:20:50.708076 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:50.707774 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" podUID="75ee05aa-3ea9-448b-9bb2-296d68a42d2e" containerName="istio-proxy" containerID="cri-o://fb48c4dec743895d93a3da2b7047216829e51fd9b8eadebe5ddd6731226d2ba1" gracePeriod=30 Jul 27 21:20:55.963520 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:55.963491 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:56.130419 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.130334 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istiod-ca-cert\") pod \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " Jul 27 21:20:56.130419 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.130372 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-data\") pod \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " Jul 27 21:20:56.130419 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.130406 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-workload-socket\") pod \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " Jul 27 21:20:56.130717 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.130424 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-token\") pod \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " Jul 27 21:20:56.130717 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.130486 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-podinfo\") pod \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " Jul 27 21:20:56.130717 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.130508 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-envoy\") pod \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " Jul 27 21:20:56.130717 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.130550 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-credential-socket\") pod \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " Jul 27 21:20:56.130717 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.130580 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-workload-certs\") pod \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " Jul 27 21:20:56.130717 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.130602 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ks4s\" (UniqueName: \"kubernetes.io/projected/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-kube-api-access-2ks4s\") pod \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\" (UID: \"75ee05aa-3ea9-448b-9bb2-296d68a42d2e\") " Jul 27 21:20:56.131004 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.130706 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "75ee05aa-3ea9-448b-9bb2-296d68a42d2e" (UID: "75ee05aa-3ea9-448b-9bb2-296d68a42d2e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:20:56.131004 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.130858 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "75ee05aa-3ea9-448b-9bb2-296d68a42d2e" (UID: "75ee05aa-3ea9-448b-9bb2-296d68a42d2e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:20:56.131004 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.130878 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "75ee05aa-3ea9-448b-9bb2-296d68a42d2e" (UID: "75ee05aa-3ea9-448b-9bb2-296d68a42d2e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:20:56.131004 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.130948 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-workload-certs\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:20:56.131004 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.130967 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-workload-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:20:56.131272 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.131174 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "75ee05aa-3ea9-448b-9bb2-296d68a42d2e" (UID: "75ee05aa-3ea9-448b-9bb2-296d68a42d2e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 21:20:56.131436 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.131414 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-data" (OuterVolumeSpecName: "istio-data") pod "75ee05aa-3ea9-448b-9bb2-296d68a42d2e" (UID: "75ee05aa-3ea9-448b-9bb2-296d68a42d2e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:20:56.132868 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.132836 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-kube-api-access-2ks4s" (OuterVolumeSpecName: "kube-api-access-2ks4s") pod "75ee05aa-3ea9-448b-9bb2-296d68a42d2e" (UID: "75ee05aa-3ea9-448b-9bb2-296d68a42d2e"). InnerVolumeSpecName "kube-api-access-2ks4s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:20:56.133012 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.132989 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "75ee05aa-3ea9-448b-9bb2-296d68a42d2e" (UID: "75ee05aa-3ea9-448b-9bb2-296d68a42d2e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:20:56.133297 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.133277 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-token" (OuterVolumeSpecName: "istio-token") pod "75ee05aa-3ea9-448b-9bb2-296d68a42d2e" (UID: "75ee05aa-3ea9-448b-9bb2-296d68a42d2e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:20:56.133367 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.133288 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "75ee05aa-3ea9-448b-9bb2-296d68a42d2e" (UID: "75ee05aa-3ea9-448b-9bb2-296d68a42d2e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 27 21:20:56.231633 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.231600 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-credential-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:20:56.231633 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.231627 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ks4s\" (UniqueName: \"kubernetes.io/projected/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-kube-api-access-2ks4s\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:20:56.231633 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.231637 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istiod-ca-cert\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:20:56.231633 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.231646 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-data\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:20:56.231907 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.231653 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-token\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:20:56.231907 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.231661 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-podinfo\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:20:56.231907 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.231669 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75ee05aa-3ea9-448b-9bb2-296d68a42d2e-istio-envoy\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:20:56.520165 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.520128 2565 generic.go:356] "Generic (PLEG): container finished" podID="75ee05aa-3ea9-448b-9bb2-296d68a42d2e" containerID="fb48c4dec743895d93a3da2b7047216829e51fd9b8eadebe5ddd6731226d2ba1" exitCode=0 Jul 27 21:20:56.520360 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.520187 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" Jul 27 21:20:56.520360 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.520220 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" event={"ID":"75ee05aa-3ea9-448b-9bb2-296d68a42d2e","Type":"ContainerDied","Data":"fb48c4dec743895d93a3da2b7047216829e51fd9b8eadebe5ddd6731226d2ba1"} Jul 27 21:20:56.520360 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.520309 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq" event={"ID":"75ee05aa-3ea9-448b-9bb2-296d68a42d2e","Type":"ContainerDied","Data":"a618769ff839eb4a8d0ac017d857274f5febbb75e9986a1b7e70f5528f693d54"} Jul 27 21:20:56.520360 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.520332 2565 scope.go:117] "RemoveContainer" containerID="fb48c4dec743895d93a3da2b7047216829e51fd9b8eadebe5ddd6731226d2ba1" Jul 27 21:20:56.529665 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.529649 2565 scope.go:117] "RemoveContainer" containerID="9f8fafab4b06a339852710bc07c67f43104b63297738963d390d7f73e113124e" Jul 27 21:20:56.538100 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.538079 2565 scope.go:117] "RemoveContainer" containerID="fb48c4dec743895d93a3da2b7047216829e51fd9b8eadebe5ddd6731226d2ba1" Jul 27 21:20:56.538386 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:20:56.538367 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb48c4dec743895d93a3da2b7047216829e51fd9b8eadebe5ddd6731226d2ba1\": container with ID starting with fb48c4dec743895d93a3da2b7047216829e51fd9b8eadebe5ddd6731226d2ba1 not found: ID does not exist" containerID="fb48c4dec743895d93a3da2b7047216829e51fd9b8eadebe5ddd6731226d2ba1" Jul 27 21:20:56.538444 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.538396 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb48c4dec743895d93a3da2b7047216829e51fd9b8eadebe5ddd6731226d2ba1"} err="failed to get container status \"fb48c4dec743895d93a3da2b7047216829e51fd9b8eadebe5ddd6731226d2ba1\": rpc error: code = NotFound desc = could not find container \"fb48c4dec743895d93a3da2b7047216829e51fd9b8eadebe5ddd6731226d2ba1\": container with ID starting with fb48c4dec743895d93a3da2b7047216829e51fd9b8eadebe5ddd6731226d2ba1 not found: ID does not exist" Jul 27 21:20:56.538444 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.538414 2565 scope.go:117] "RemoveContainer" containerID="9f8fafab4b06a339852710bc07c67f43104b63297738963d390d7f73e113124e" Jul 27 21:20:56.538669 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:20:56.538655 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f8fafab4b06a339852710bc07c67f43104b63297738963d390d7f73e113124e\": container with ID starting with 9f8fafab4b06a339852710bc07c67f43104b63297738963d390d7f73e113124e not found: ID does not exist" containerID="9f8fafab4b06a339852710bc07c67f43104b63297738963d390d7f73e113124e" Jul 27 21:20:56.538706 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.538673 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f8fafab4b06a339852710bc07c67f43104b63297738963d390d7f73e113124e"} err="failed to get container status \"9f8fafab4b06a339852710bc07c67f43104b63297738963d390d7f73e113124e\": rpc error: code = NotFound desc = could not find container \"9f8fafab4b06a339852710bc07c67f43104b63297738963d390d7f73e113124e\": container with ID starting with 9f8fafab4b06a339852710bc07c67f43104b63297738963d390d7f73e113124e not found: ID does not exist" Jul 27 21:20:56.541794 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.541773 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq"] Jul 27 21:20:56.545947 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:56.545927 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-a9503936-openshift-default-cbf98cf56-fqvdq"] Jul 27 21:20:58.053059 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:20:58.053023 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75ee05aa-3ea9-448b-9bb2-296d68a42d2e" path="/var/lib/kubelet/pods/75ee05aa-3ea9-448b-9bb2-296d68a42d2e/volumes" Jul 27 21:22:41.245383 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.245304 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb"] Jul 27 21:22:41.245843 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.245670 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf226b23-266c-4e59-bc7e-e8fb019bbf52" containerName="istio-proxy" Jul 27 21:22:41.245843 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.245684 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf226b23-266c-4e59-bc7e-e8fb019bbf52" containerName="istio-proxy" Jul 27 21:22:41.245843 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.245704 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf226b23-266c-4e59-bc7e-e8fb019bbf52" containerName="istio-proxy" Jul 27 21:22:41.245843 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.245710 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf226b23-266c-4e59-bc7e-e8fb019bbf52" containerName="istio-proxy" Jul 27 21:22:41.245843 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.245720 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75ee05aa-3ea9-448b-9bb2-296d68a42d2e" containerName="istio-proxy" Jul 27 21:22:41.245843 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.245726 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="75ee05aa-3ea9-448b-9bb2-296d68a42d2e" containerName="istio-proxy" Jul 27 21:22:41.245843 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.245732 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75ee05aa-3ea9-448b-9bb2-296d68a42d2e" containerName="istio-proxy" Jul 27 21:22:41.245843 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.245737 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="75ee05aa-3ea9-448b-9bb2-296d68a42d2e" containerName="istio-proxy" Jul 27 21:22:41.245843 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.245803 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf226b23-266c-4e59-bc7e-e8fb019bbf52" containerName="istio-proxy" Jul 27 21:22:41.245843 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.245811 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="75ee05aa-3ea9-448b-9bb2-296d68a42d2e" containerName="istio-proxy" Jul 27 21:22:41.245843 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.245816 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf226b23-266c-4e59-bc7e-e8fb019bbf52" containerName="istio-proxy" Jul 27 21:22:41.245843 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.245823 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="75ee05aa-3ea9-448b-9bb2-296d68a42d2e" containerName="istio-proxy" Jul 27 21:22:41.248896 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.248878 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.251693 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.251672 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-61de35ca-openshift-default-dockercfg-vpdqg\"" Jul 27 21:22:41.270535 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.270507 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb"] Jul 27 21:22:41.363894 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.363865 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-workload-socket\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.364073 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.363901 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-envoy\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.364073 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.363929 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istiod-ca-cert\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.364073 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.363962 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-data\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.364073 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.363978 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2fj2\" (UniqueName: \"kubernetes.io/projected/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-kube-api-access-p2fj2\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.364073 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.364000 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-workload-certs\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.364073 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.364039 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-credential-socket\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.364073 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.364060 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-podinfo\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.364361 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.364080 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-token\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.465101 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.465067 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-credential-socket\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.465301 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.465111 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-podinfo\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.465301 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.465141 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-token\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.465406 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.465311 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-workload-socket\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.465406 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.465345 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-envoy\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.465406 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.465369 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istiod-ca-cert\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.465591 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.465568 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-credential-socket\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.465772 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.465737 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-workload-socket\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.466065 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.465757 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-data\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.466065 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.465816 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2fj2\" (UniqueName: \"kubernetes.io/projected/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-kube-api-access-p2fj2\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.466065 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.465854 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-workload-certs\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.466288 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.466070 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-data\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.466288 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.466142 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istiod-ca-cert\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.466288 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.466258 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-workload-certs\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.467739 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.467712 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-envoy\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.467848 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.467830 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-podinfo\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.473760 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.473738 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2fj2\" (UniqueName: \"kubernetes.io/projected/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-kube-api-access-p2fj2\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.473854 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.473791 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-token\") pod \"e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.558586 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.558492 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:41.692802 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.692770 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb"] Jul 27 21:22:41.693396 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:22:41.693367 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96f5e50c_75f3_4cfc_b316_bd24f8a808a1.slice/crio-bdaa8020c6882146d29d3d0003be31a7b576fc85c052430e7d09ec00d4cc0075 WatchSource:0}: Error finding container bdaa8020c6882146d29d3d0003be31a7b576fc85c052430e7d09ec00d4cc0075: Status 404 returned error can't find the container with id bdaa8020c6882146d29d3d0003be31a7b576fc85c052430e7d09ec00d4cc0075 Jul 27 21:22:41.695651 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.695618 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:22:41.695744 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.695684 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:22:41.695744 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:41.695713 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:22:42.051335 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:42.051300 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" event={"ID":"96f5e50c-75f3-4cfc-b316-bd24f8a808a1","Type":"ContainerStarted","Data":"2cfae6613c01d354ea23c536246f3051c65e661636029ff3990c24d34201ca0d"} Jul 27 21:22:42.051505 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:42.051346 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" event={"ID":"96f5e50c-75f3-4cfc-b316-bd24f8a808a1","Type":"ContainerStarted","Data":"bdaa8020c6882146d29d3d0003be31a7b576fc85c052430e7d09ec00d4cc0075"} Jul 27 21:22:42.070086 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:42.070038 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" podStartSLOduration=1.070024772 podStartE2EDuration="1.070024772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:22:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:22:42.068169686 +0000 UTC m=+1890.591287406" watchObservedRunningTime="2026-07-27 21:22:42.070024772 +0000 UTC m=+1890.593142493" Jul 27 21:22:42.559405 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:42.559373 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:42.564460 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:42.564432 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:43.052274 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:43.052229 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:43.053122 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:43.053106 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:58.054065 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:58.054035 2565 patch_prober.go:28] interesting pod/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 27 21:22:58.054496 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:58.054121 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" podUID="96f5e50c-75f3-4cfc-b316-bd24f8a808a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 27 21:22:58.126220 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:58.126180 2565 generic.go:356] "Generic (PLEG): container finished" podID="96f5e50c-75f3-4cfc-b316-bd24f8a808a1" containerID="2cfae6613c01d354ea23c536246f3051c65e661636029ff3990c24d34201ca0d" exitCode=0 Jul 27 21:22:58.126408 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:58.126257 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" event={"ID":"96f5e50c-75f3-4cfc-b316-bd24f8a808a1","Type":"ContainerDied","Data":"2cfae6613c01d354ea23c536246f3051c65e661636029ff3990c24d34201ca0d"} Jul 27 21:22:58.126634 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:58.126615 2565 scope.go:117] "RemoveContainer" containerID="2cfae6613c01d354ea23c536246f3051c65e661636029ff3990c24d34201ca0d" Jul 27 21:22:58.127615 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:58.127589 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:22:58.127668 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:58.127659 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:22:58.127706 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:58.127690 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:22:59.133356 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:59.133323 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" event={"ID":"96f5e50c-75f3-4cfc-b316-bd24f8a808a1","Type":"ContainerStarted","Data":"9351469c3769d86398c4fe521fe04b6b99303d52b18acf7edb188e3747a2588c"} Jul 27 21:22:59.558992 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:59.558952 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:22:59.559206 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:59.559183 2565 patch_prober.go:28] interesting pod/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 27 21:22:59.559280 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:22:59.559259 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" podUID="96f5e50c-75f3-4cfc-b316-bd24f8a808a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 27 21:23:00.558927 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:00.558890 2565 patch_prober.go:28] interesting pod/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 27 21:23:00.559354 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:00.558943 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" podUID="96f5e50c-75f3-4cfc-b316-bd24f8a808a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 27 21:23:01.558997 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:01.558963 2565 patch_prober.go:28] interesting pod/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 27 21:23:01.559387 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:01.559026 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" podUID="96f5e50c-75f3-4cfc-b316-bd24f8a808a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 27 21:23:02.559303 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:02.559265 2565 patch_prober.go:28] interesting pod/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 27 21:23:02.559798 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:02.559333 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" podUID="96f5e50c-75f3-4cfc-b316-bd24f8a808a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 27 21:23:03.562833 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:03.562803 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:23:03.563347 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:03.563176 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:23:03.563953 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:03.563934 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:23:17.492181 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:17.492135 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb"] Jul 27 21:23:17.492754 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:17.492426 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" podUID="96f5e50c-75f3-4cfc-b316-bd24f8a808a1" containerName="istio-proxy" containerID="cri-o://9351469c3769d86398c4fe521fe04b6b99303d52b18acf7edb188e3747a2588c" gracePeriod=30 Jul 27 21:23:18.565702 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:18.565662 2565 patch_prober.go:28] interesting pod/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 27 21:23:18.566069 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:18.565736 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" podUID="96f5e50c-75f3-4cfc-b316-bd24f8a808a1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 27 21:23:22.742930 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.742906 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:23:22.832815 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.832707 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-workload-socket\") pod \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " Jul 27 21:23:22.832815 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.832765 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-podinfo\") pod \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " Jul 27 21:23:22.833051 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.832838 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-workload-certs\") pod \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " Jul 27 21:23:22.833051 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.832879 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-credential-socket\") pod \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " Jul 27 21:23:22.833051 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.832910 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istiod-ca-cert\") pod \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " Jul 27 21:23:22.833051 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.832950 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-data\") pod \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " Jul 27 21:23:22.833051 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.832976 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-token\") pod \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " Jul 27 21:23:22.833051 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.833024 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-envoy\") pod \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " Jul 27 21:23:22.833051 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.833023 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "96f5e50c-75f3-4cfc-b316-bd24f8a808a1" (UID: "96f5e50c-75f3-4cfc-b316-bd24f8a808a1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:23:22.833446 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.833047 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2fj2\" (UniqueName: \"kubernetes.io/projected/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-kube-api-access-p2fj2\") pod \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\" (UID: \"96f5e50c-75f3-4cfc-b316-bd24f8a808a1\") " Jul 27 21:23:22.833446 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.833230 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "96f5e50c-75f3-4cfc-b316-bd24f8a808a1" (UID: "96f5e50c-75f3-4cfc-b316-bd24f8a808a1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:23:22.833446 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.833384 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "96f5e50c-75f3-4cfc-b316-bd24f8a808a1" (UID: "96f5e50c-75f3-4cfc-b316-bd24f8a808a1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:23:22.833565 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.833485 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "96f5e50c-75f3-4cfc-b316-bd24f8a808a1" (UID: "96f5e50c-75f3-4cfc-b316-bd24f8a808a1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 21:23:22.833565 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.833499 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-workload-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:23:22.833565 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.833519 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-workload-certs\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:23:22.833565 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.833533 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-credential-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:23:22.834054 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.834035 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-data" (OuterVolumeSpecName: "istio-data") pod "96f5e50c-75f3-4cfc-b316-bd24f8a808a1" (UID: "96f5e50c-75f3-4cfc-b316-bd24f8a808a1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:23:22.835847 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.835816 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "96f5e50c-75f3-4cfc-b316-bd24f8a808a1" (UID: "96f5e50c-75f3-4cfc-b316-bd24f8a808a1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 27 21:23:22.835847 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.835826 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "96f5e50c-75f3-4cfc-b316-bd24f8a808a1" (UID: "96f5e50c-75f3-4cfc-b316-bd24f8a808a1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:23:22.835847 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.835824 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-kube-api-access-p2fj2" (OuterVolumeSpecName: "kube-api-access-p2fj2") pod "96f5e50c-75f3-4cfc-b316-bd24f8a808a1" (UID: "96f5e50c-75f3-4cfc-b316-bd24f8a808a1"). InnerVolumeSpecName "kube-api-access-p2fj2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:23:22.836017 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.835866 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-token" (OuterVolumeSpecName: "istio-token") pod "96f5e50c-75f3-4cfc-b316-bd24f8a808a1" (UID: "96f5e50c-75f3-4cfc-b316-bd24f8a808a1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:23:22.934895 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.934839 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-data\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:23:22.934895 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.934885 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-token\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:23:22.934895 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.934894 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-envoy\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:23:22.934895 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.934904 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2fj2\" (UniqueName: \"kubernetes.io/projected/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-kube-api-access-p2fj2\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:23:22.934895 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.934915 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istio-podinfo\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:23:22.935209 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:22.934923 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96f5e50c-75f3-4cfc-b316-bd24f8a808a1-istiod-ca-cert\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:23:23.255657 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:23.255621 2565 generic.go:356] "Generic (PLEG): container finished" podID="96f5e50c-75f3-4cfc-b316-bd24f8a808a1" containerID="9351469c3769d86398c4fe521fe04b6b99303d52b18acf7edb188e3747a2588c" exitCode=0 Jul 27 21:23:23.255826 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:23.255688 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" Jul 27 21:23:23.255826 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:23.255706 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" event={"ID":"96f5e50c-75f3-4cfc-b316-bd24f8a808a1","Type":"ContainerDied","Data":"9351469c3769d86398c4fe521fe04b6b99303d52b18acf7edb188e3747a2588c"} Jul 27 21:23:23.255826 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:23.255765 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb" event={"ID":"96f5e50c-75f3-4cfc-b316-bd24f8a808a1","Type":"ContainerDied","Data":"bdaa8020c6882146d29d3d0003be31a7b576fc85c052430e7d09ec00d4cc0075"} Jul 27 21:23:23.255826 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:23.255790 2565 scope.go:117] "RemoveContainer" containerID="9351469c3769d86398c4fe521fe04b6b99303d52b18acf7edb188e3747a2588c" Jul 27 21:23:23.265027 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:23.265007 2565 scope.go:117] "RemoveContainer" containerID="2cfae6613c01d354ea23c536246f3051c65e661636029ff3990c24d34201ca0d" Jul 27 21:23:23.273317 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:23.273299 2565 scope.go:117] "RemoveContainer" containerID="9351469c3769d86398c4fe521fe04b6b99303d52b18acf7edb188e3747a2588c" Jul 27 21:23:23.273575 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:23:23.273556 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9351469c3769d86398c4fe521fe04b6b99303d52b18acf7edb188e3747a2588c\": container with ID starting with 9351469c3769d86398c4fe521fe04b6b99303d52b18acf7edb188e3747a2588c not found: ID does not exist" containerID="9351469c3769d86398c4fe521fe04b6b99303d52b18acf7edb188e3747a2588c" Jul 27 21:23:23.273624 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:23.273582 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9351469c3769d86398c4fe521fe04b6b99303d52b18acf7edb188e3747a2588c"} err="failed to get container status \"9351469c3769d86398c4fe521fe04b6b99303d52b18acf7edb188e3747a2588c\": rpc error: code = NotFound desc = could not find container \"9351469c3769d86398c4fe521fe04b6b99303d52b18acf7edb188e3747a2588c\": container with ID starting with 9351469c3769d86398c4fe521fe04b6b99303d52b18acf7edb188e3747a2588c not found: ID does not exist" Jul 27 21:23:23.273624 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:23.273598 2565 scope.go:117] "RemoveContainer" containerID="2cfae6613c01d354ea23c536246f3051c65e661636029ff3990c24d34201ca0d" Jul 27 21:23:23.273844 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:23:23.273825 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cfae6613c01d354ea23c536246f3051c65e661636029ff3990c24d34201ca0d\": container with ID starting with 2cfae6613c01d354ea23c536246f3051c65e661636029ff3990c24d34201ca0d not found: ID does not exist" containerID="2cfae6613c01d354ea23c536246f3051c65e661636029ff3990c24d34201ca0d" Jul 27 21:23:23.273898 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:23.273850 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cfae6613c01d354ea23c536246f3051c65e661636029ff3990c24d34201ca0d"} err="failed to get container status \"2cfae6613c01d354ea23c536246f3051c65e661636029ff3990c24d34201ca0d\": rpc error: code = NotFound desc = could not find container \"2cfae6613c01d354ea23c536246f3051c65e661636029ff3990c24d34201ca0d\": container with ID starting with 2cfae6613c01d354ea23c536246f3051c65e661636029ff3990c24d34201ca0d not found: ID does not exist" Jul 27 21:23:23.278651 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:23.278629 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb"] Jul 27 21:23:23.283471 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:23.283450 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-61de35ca-openshift-default-5b6fbc886d-gd6sb"] Jul 27 21:23:24.052116 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:24.052087 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96f5e50c-75f3-4cfc-b316-bd24f8a808a1" path="/var/lib/kubelet/pods/96f5e50c-75f3-4cfc-b316-bd24f8a808a1/volumes" Jul 27 21:23:37.416612 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.416579 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv"] Jul 27 21:23:37.416987 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.416933 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96f5e50c-75f3-4cfc-b316-bd24f8a808a1" containerName="istio-proxy" Jul 27 21:23:37.416987 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.416945 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="96f5e50c-75f3-4cfc-b316-bd24f8a808a1" containerName="istio-proxy" Jul 27 21:23:37.416987 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.416964 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96f5e50c-75f3-4cfc-b316-bd24f8a808a1" containerName="istio-proxy" Jul 27 21:23:37.416987 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.416970 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="96f5e50c-75f3-4cfc-b316-bd24f8a808a1" containerName="istio-proxy" Jul 27 21:23:37.417208 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.417029 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="96f5e50c-75f3-4cfc-b316-bd24f8a808a1" containerName="istio-proxy" Jul 27 21:23:37.420614 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.420592 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.423427 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.423409 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-316c1e32-openshift-default-dockercfg-gk5t7\"" Jul 27 21:23:37.431952 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.431926 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv"] Jul 27 21:23:37.468508 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.468469 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-token\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.468668 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.468516 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istiod-ca-cert\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.468668 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.468537 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9pd5\" (UniqueName: \"kubernetes.io/projected/43292ee5-cd92-4cad-be70-d4e8c217cc0d-kube-api-access-k9pd5\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.468668 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.468617 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-envoy\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.468806 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.468694 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-data\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.468806 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.468726 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-workload-socket\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.468806 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.468770 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-workload-certs\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.468806 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.468795 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-podinfo\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.468947 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.468829 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-credential-socket\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.569693 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.569657 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istiod-ca-cert\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.569693 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.569698 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9pd5\" (UniqueName: \"kubernetes.io/projected/43292ee5-cd92-4cad-be70-d4e8c217cc0d-kube-api-access-k9pd5\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.569951 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.569718 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-envoy\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.569951 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.569756 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-data\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.569951 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.569777 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-workload-socket\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.569951 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.569808 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-workload-certs\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.569951 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.569823 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-podinfo\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.569951 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.569844 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-credential-socket\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.569951 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.569894 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-token\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.570350 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.570280 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-workload-certs\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.570417 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.570354 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-workload-socket\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.570417 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.570375 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-data\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.570494 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.570449 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-credential-socket\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.570965 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.570943 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istiod-ca-cert\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.572395 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.572374 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-envoy\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.572513 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.572495 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-podinfo\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.578091 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.578068 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-token\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.578269 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.578252 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9pd5\" (UniqueName: \"kubernetes.io/projected/43292ee5-cd92-4cad-be70-d4e8c217cc0d-kube-api-access-k9pd5\") pod \"e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.731199 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.731087 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:37.866803 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.866768 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv"] Jul 27 21:23:37.868102 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:23:37.868076 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43292ee5_cd92_4cad_be70_d4e8c217cc0d.slice/crio-ce24f27b1860f7a7af6a1147fdd0f00712fde62794285233f644b40796316527 WatchSource:0}: Error finding container ce24f27b1860f7a7af6a1147fdd0f00712fde62794285233f644b40796316527: Status 404 returned error can't find the container with id ce24f27b1860f7a7af6a1147fdd0f00712fde62794285233f644b40796316527 Jul 27 21:23:37.870409 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.870371 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:23:37.870494 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.870455 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:23:37.870535 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:37.870499 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:23:38.329622 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:38.329584 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" event={"ID":"43292ee5-cd92-4cad-be70-d4e8c217cc0d","Type":"ContainerStarted","Data":"ad114f55abcab5391417f50441c04f2c49ee991ba2b64e917029c8270b981c08"} Jul 27 21:23:38.329826 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:38.329638 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" event={"ID":"43292ee5-cd92-4cad-be70-d4e8c217cc0d","Type":"ContainerStarted","Data":"ce24f27b1860f7a7af6a1147fdd0f00712fde62794285233f644b40796316527"} Jul 27 21:23:38.349985 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:38.349938 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" podStartSLOduration=1.349920306 podStartE2EDuration="1.349920306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:23:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:23:38.347388488 +0000 UTC m=+1946.870506210" watchObservedRunningTime="2026-07-27 21:23:38.349920306 +0000 UTC m=+1946.873038028" Jul 27 21:23:38.731653 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:38.731613 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:38.736431 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:38.736405 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:39.335164 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:39.335132 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:39.336168 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:39.336149 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:53.411570 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:53.411473 2565 generic.go:356] "Generic (PLEG): container finished" podID="43292ee5-cd92-4cad-be70-d4e8c217cc0d" containerID="ad114f55abcab5391417f50441c04f2c49ee991ba2b64e917029c8270b981c08" exitCode=0 Jul 27 21:23:53.411570 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:53.411547 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" event={"ID":"43292ee5-cd92-4cad-be70-d4e8c217cc0d","Type":"ContainerDied","Data":"ad114f55abcab5391417f50441c04f2c49ee991ba2b64e917029c8270b981c08"} Jul 27 21:23:53.412008 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:53.411931 2565 scope.go:117] "RemoveContainer" containerID="ad114f55abcab5391417f50441c04f2c49ee991ba2b64e917029c8270b981c08" Jul 27 21:23:53.412916 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:53.412885 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:23:53.413013 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:53.412959 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:23:53.413013 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:53.412989 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:23:54.335074 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:54.335033 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:54.418692 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:54.418654 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" event={"ID":"43292ee5-cd92-4cad-be70-d4e8c217cc0d","Type":"ContainerStarted","Data":"445a43c06983d7f8029805544e172a7186e5e13fa33a4bf3e45728bc606c97dd"} Jul 27 21:23:54.732229 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:54.732186 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:54.733285 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:54.733224 2565 patch_prober.go:28] interesting pod/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 27 21:23:54.733416 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:54.733314 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" podUID="43292ee5-cd92-4cad-be70-d4e8c217cc0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 27 21:23:55.732060 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:55.732021 2565 patch_prober.go:28] interesting pod/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 27 21:23:55.732499 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:55.732087 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" podUID="43292ee5-cd92-4cad-be70-d4e8c217cc0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 27 21:23:56.732175 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:56.732139 2565 patch_prober.go:28] interesting pod/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 27 21:23:56.732611 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:56.732195 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" podUID="43292ee5-cd92-4cad-be70-d4e8c217cc0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 27 21:23:57.731651 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:57.731608 2565 patch_prober.go:28] interesting pod/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 27 21:23:57.731831 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:57.731697 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" podUID="43292ee5-cd92-4cad-be70-d4e8c217cc0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 27 21:23:58.735802 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:58.735768 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:58.736311 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:58.736185 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:23:58.736756 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:23:58.736737 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:25:40.630794 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:40.630710 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv"] Jul 27 21:25:40.631408 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:40.630958 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" podUID="43292ee5-cd92-4cad-be70-d4e8c217cc0d" containerName="istio-proxy" containerID="cri-o://445a43c06983d7f8029805544e172a7186e5e13fa33a4bf3e45728bc606c97dd" gracePeriod=30 Jul 27 21:25:43.738659 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:43.738625 2565 patch_prober.go:28] interesting pod/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 27 21:25:43.739019 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:43.738680 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" podUID="43292ee5-cd92-4cad-be70-d4e8c217cc0d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 27 21:25:45.892665 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.892595 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:25:45.971080 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.971035 2565 generic.go:356] "Generic (PLEG): container finished" podID="43292ee5-cd92-4cad-be70-d4e8c217cc0d" containerID="445a43c06983d7f8029805544e172a7186e5e13fa33a4bf3e45728bc606c97dd" exitCode=0 Jul 27 21:25:45.971261 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.971152 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" Jul 27 21:25:45.971261 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.971164 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" event={"ID":"43292ee5-cd92-4cad-be70-d4e8c217cc0d","Type":"ContainerDied","Data":"445a43c06983d7f8029805544e172a7186e5e13fa33a4bf3e45728bc606c97dd"} Jul 27 21:25:45.971261 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.971229 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv" event={"ID":"43292ee5-cd92-4cad-be70-d4e8c217cc0d","Type":"ContainerDied","Data":"ce24f27b1860f7a7af6a1147fdd0f00712fde62794285233f644b40796316527"} Jul 27 21:25:45.971382 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.971268 2565 scope.go:117] "RemoveContainer" containerID="445a43c06983d7f8029805544e172a7186e5e13fa33a4bf3e45728bc606c97dd" Jul 27 21:25:45.980621 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.980598 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-podinfo\") pod \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " Jul 27 21:25:45.980769 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.980633 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-workload-socket\") pod \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " Jul 27 21:25:45.980769 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.980684 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-token\") pod \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " Jul 27 21:25:45.980769 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.980721 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-credential-socket\") pod \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " Jul 27 21:25:45.980930 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.980845 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istiod-ca-cert\") pod \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " Jul 27 21:25:45.980984 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.980938 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-envoy\") pod \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " Jul 27 21:25:45.980984 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.980968 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9pd5\" (UniqueName: \"kubernetes.io/projected/43292ee5-cd92-4cad-be70-d4e8c217cc0d-kube-api-access-k9pd5\") pod \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " Jul 27 21:25:45.981088 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.980994 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-workload-certs\") pod \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " Jul 27 21:25:45.981088 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.981028 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-data\") pod \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\" (UID: \"43292ee5-cd92-4cad-be70-d4e8c217cc0d\") " Jul 27 21:25:45.981088 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.981021 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "43292ee5-cd92-4cad-be70-d4e8c217cc0d" (UID: "43292ee5-cd92-4cad-be70-d4e8c217cc0d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:25:45.981088 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.981030 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "43292ee5-cd92-4cad-be70-d4e8c217cc0d" (UID: "43292ee5-cd92-4cad-be70-d4e8c217cc0d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:25:45.981853 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.981358 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-workload-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:25:45.981853 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.981387 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-credential-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:25:45.981853 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.981626 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "43292ee5-cd92-4cad-be70-d4e8c217cc0d" (UID: "43292ee5-cd92-4cad-be70-d4e8c217cc0d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:25:45.981853 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.981813 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "43292ee5-cd92-4cad-be70-d4e8c217cc0d" (UID: "43292ee5-cd92-4cad-be70-d4e8c217cc0d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 21:25:45.982499 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.982398 2565 scope.go:117] "RemoveContainer" containerID="ad114f55abcab5391417f50441c04f2c49ee991ba2b64e917029c8270b981c08" Jul 27 21:25:45.982499 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.982413 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-data" (OuterVolumeSpecName: "istio-data") pod "43292ee5-cd92-4cad-be70-d4e8c217cc0d" (UID: "43292ee5-cd92-4cad-be70-d4e8c217cc0d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:25:45.983524 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.983490 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43292ee5-cd92-4cad-be70-d4e8c217cc0d-kube-api-access-k9pd5" (OuterVolumeSpecName: "kube-api-access-k9pd5") pod "43292ee5-cd92-4cad-be70-d4e8c217cc0d" (UID: "43292ee5-cd92-4cad-be70-d4e8c217cc0d"). InnerVolumeSpecName "kube-api-access-k9pd5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:25:45.983524 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.983511 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-token" (OuterVolumeSpecName: "istio-token") pod "43292ee5-cd92-4cad-be70-d4e8c217cc0d" (UID: "43292ee5-cd92-4cad-be70-d4e8c217cc0d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:25:45.983688 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.983535 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "43292ee5-cd92-4cad-be70-d4e8c217cc0d" (UID: "43292ee5-cd92-4cad-be70-d4e8c217cc0d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 27 21:25:45.983688 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:45.983624 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "43292ee5-cd92-4cad-be70-d4e8c217cc0d" (UID: "43292ee5-cd92-4cad-be70-d4e8c217cc0d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:25:46.001615 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:46.001595 2565 scope.go:117] "RemoveContainer" containerID="445a43c06983d7f8029805544e172a7186e5e13fa33a4bf3e45728bc606c97dd" Jul 27 21:25:46.001949 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:25:46.001928 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"445a43c06983d7f8029805544e172a7186e5e13fa33a4bf3e45728bc606c97dd\": container with ID starting with 445a43c06983d7f8029805544e172a7186e5e13fa33a4bf3e45728bc606c97dd not found: ID does not exist" containerID="445a43c06983d7f8029805544e172a7186e5e13fa33a4bf3e45728bc606c97dd" Jul 27 21:25:46.002002 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:46.001961 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"445a43c06983d7f8029805544e172a7186e5e13fa33a4bf3e45728bc606c97dd"} err="failed to get container status \"445a43c06983d7f8029805544e172a7186e5e13fa33a4bf3e45728bc606c97dd\": rpc error: code = NotFound desc = could not find container \"445a43c06983d7f8029805544e172a7186e5e13fa33a4bf3e45728bc606c97dd\": container with ID starting with 445a43c06983d7f8029805544e172a7186e5e13fa33a4bf3e45728bc606c97dd not found: ID does not exist" Jul 27 21:25:46.002002 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:46.001982 2565 scope.go:117] "RemoveContainer" containerID="ad114f55abcab5391417f50441c04f2c49ee991ba2b64e917029c8270b981c08" Jul 27 21:25:46.002253 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:25:46.002216 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad114f55abcab5391417f50441c04f2c49ee991ba2b64e917029c8270b981c08\": container with ID starting with ad114f55abcab5391417f50441c04f2c49ee991ba2b64e917029c8270b981c08 not found: ID does not exist" containerID="ad114f55abcab5391417f50441c04f2c49ee991ba2b64e917029c8270b981c08" Jul 27 21:25:46.002324 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:46.002252 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad114f55abcab5391417f50441c04f2c49ee991ba2b64e917029c8270b981c08"} err="failed to get container status \"ad114f55abcab5391417f50441c04f2c49ee991ba2b64e917029c8270b981c08\": rpc error: code = NotFound desc = could not find container \"ad114f55abcab5391417f50441c04f2c49ee991ba2b64e917029c8270b981c08\": container with ID starting with ad114f55abcab5391417f50441c04f2c49ee991ba2b64e917029c8270b981c08 not found: ID does not exist" Jul 27 21:25:46.082885 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:46.082756 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istiod-ca-cert\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:25:46.082885 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:46.082793 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-envoy\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:25:46.082885 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:46.082806 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9pd5\" (UniqueName: \"kubernetes.io/projected/43292ee5-cd92-4cad-be70-d4e8c217cc0d-kube-api-access-k9pd5\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:25:46.082885 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:46.082821 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-workload-certs\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:25:46.082885 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:46.082835 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-data\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:25:46.082885 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:46.082847 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-podinfo\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:25:46.082885 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:46.082858 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43292ee5-cd92-4cad-be70-d4e8c217cc0d-istio-token\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:25:46.288781 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:46.288749 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv"] Jul 27 21:25:46.290574 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:46.290552 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-316c1e32-openshift-default-7567c7ffc6-2flfv"] Jul 27 21:25:48.052890 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:25:48.052858 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43292ee5-cd92-4cad-be70-d4e8c217cc0d" path="/var/lib/kubelet/pods/43292ee5-cd92-4cad-be70-d4e8c217cc0d/volumes" Jul 27 21:26:35.214802 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.214770 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv"] Jul 27 21:26:35.215188 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.215109 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43292ee5-cd92-4cad-be70-d4e8c217cc0d" containerName="istio-proxy" Jul 27 21:26:35.215188 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.215120 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="43292ee5-cd92-4cad-be70-d4e8c217cc0d" containerName="istio-proxy" Jul 27 21:26:35.215188 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.215133 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43292ee5-cd92-4cad-be70-d4e8c217cc0d" containerName="istio-proxy" Jul 27 21:26:35.215188 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.215138 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="43292ee5-cd92-4cad-be70-d4e8c217cc0d" containerName="istio-proxy" Jul 27 21:26:35.215356 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.215192 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="96f5e50c-75f3-4cfc-b316-bd24f8a808a1" containerName="istio-proxy" Jul 27 21:26:35.215356 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.215200 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="43292ee5-cd92-4cad-be70-d4e8c217cc0d" containerName="istio-proxy" Jul 27 21:26:35.218018 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.218002 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.221004 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.220984 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-edfab0-openshift-default-dockercfg-wjkqm\"" Jul 27 21:26:35.233920 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.233898 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv"] Jul 27 21:26:35.313038 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.313001 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-envoy\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.313262 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.313043 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-podinfo\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.313262 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.313084 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-workload-socket\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.313262 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.313105 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-credential-socket\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.313262 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.313128 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msjq5\" (UniqueName: \"kubernetes.io/projected/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-kube-api-access-msjq5\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.313262 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.313227 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-token\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.313496 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.313300 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-workload-certs\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.313496 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.313339 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istiod-ca-cert\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.313496 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.313370 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-data\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.414408 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.414368 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-workload-certs\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.414408 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.414410 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istiod-ca-cert\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.414678 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.414429 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-data\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.414678 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.414464 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-envoy\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.414678 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.414490 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-podinfo\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.414678 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.414522 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-workload-socket\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.414678 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.414548 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-credential-socket\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.414678 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.414571 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msjq5\" (UniqueName: \"kubernetes.io/projected/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-kube-api-access-msjq5\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.414678 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.414629 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-token\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.415036 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.414811 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-workload-certs\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.415036 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.414988 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-credential-socket\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.415142 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.415032 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-workload-socket\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.415142 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.415102 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-data\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.415300 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.415183 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istiod-ca-cert\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.417407 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.417383 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-envoy\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.417670 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.417643 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-podinfo\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.422967 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.422946 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-token\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.423122 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.423099 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msjq5\" (UniqueName: \"kubernetes.io/projected/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-kube-api-access-msjq5\") pod \"e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.527777 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.527686 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:35.666142 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.666104 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv"] Jul 27 21:26:35.667379 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:26:35.667349 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc06b3fe_69d1_4311_ade0_ee4e0dbdb8e2.slice/crio-4cb59414f2cab4f2f5b3975fb0e15120739a2d957964d7b22b7f08c687303f5e WatchSource:0}: Error finding container 4cb59414f2cab4f2f5b3975fb0e15120739a2d957964d7b22b7f08c687303f5e: Status 404 returned error can't find the container with id 4cb59414f2cab4f2f5b3975fb0e15120739a2d957964d7b22b7f08c687303f5e Jul 27 21:26:35.669779 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.669747 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:26:35.669850 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.669819 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:26:35.669891 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:35.669853 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:26:36.219884 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:36.219849 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" event={"ID":"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2","Type":"ContainerStarted","Data":"7e4aca0ed67b6a960ac1c7f045eee55185b07e82c4a502eff603dacccdef1182"} Jul 27 21:26:36.220300 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:36.219906 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" event={"ID":"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2","Type":"ContainerStarted","Data":"4cb59414f2cab4f2f5b3975fb0e15120739a2d957964d7b22b7f08c687303f5e"} Jul 27 21:26:36.269183 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:36.269112 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" podStartSLOduration=1.269093145 podStartE2EDuration="1.269093145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:26:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:26:36.25972518 +0000 UTC m=+2124.782842902" watchObservedRunningTime="2026-07-27 21:26:36.269093145 +0000 UTC m=+2124.792211129" Jul 27 21:26:36.528723 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:36.528635 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:36.533535 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:36.533512 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:37.225005 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:37.224976 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:37.225804 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:37.225784 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:43.353032 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.352998 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx"] Jul 27 21:26:43.353490 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.353444 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="43292ee5-cd92-4cad-be70-d4e8c217cc0d" containerName="istio-proxy" Jul 27 21:26:43.355727 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.355711 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.358844 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.358825 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-0adc64-openshift-default-dockercfg-gph5s\"" Jul 27 21:26:43.390852 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.385898 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx"] Jul 27 21:26:43.491405 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.491356 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/267f2d48-d276-4d69-a9ae-053326ef42f6-istiod-ca-cert\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.491624 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.491431 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcd7m\" (UniqueName: \"kubernetes.io/projected/267f2d48-d276-4d69-a9ae-053326ef42f6-kube-api-access-dcd7m\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.491624 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.491485 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-data\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.491624 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.491574 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-podinfo\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.491624 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.491621 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-workload-certs\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.491822 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.491646 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-token\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.491822 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.491692 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-credential-socket\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.491822 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.491751 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-envoy\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.491822 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.491784 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-workload-socket\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.592764 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.592723 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-workload-socket\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.592960 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.592789 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/267f2d48-d276-4d69-a9ae-053326ef42f6-istiod-ca-cert\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.592960 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.592814 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcd7m\" (UniqueName: \"kubernetes.io/projected/267f2d48-d276-4d69-a9ae-053326ef42f6-kube-api-access-dcd7m\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.592960 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.592834 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-data\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.592960 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.592872 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-podinfo\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.592960 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.592902 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-workload-certs\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.592960 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.592923 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-token\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.593327 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.593106 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-credential-socket\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.593327 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.593167 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-envoy\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.593327 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.593281 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-workload-socket\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.593483 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.593415 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-data\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.593483 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.593438 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-workload-certs\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.593553 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.593536 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-credential-socket\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.593644 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.593619 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/267f2d48-d276-4d69-a9ae-053326ef42f6-istiod-ca-cert\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.595467 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.595442 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-envoy\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.595669 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.595650 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-podinfo\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.602032 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.602011 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-token\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.603962 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.603896 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcd7m\" (UniqueName: \"kubernetes.io/projected/267f2d48-d276-4d69-a9ae-053326ef42f6-kube-api-access-dcd7m\") pod \"e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.686205 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.686161 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:43.855478 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.855450 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx"] Jul 27 21:26:43.857003 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:26:43.856980 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod267f2d48_d276_4d69_a9ae_053326ef42f6.slice/crio-8a6628ea46b6f6882f9cde34278e7091ce538bb0fa31620ccac79179239f0f0e WatchSource:0}: Error finding container 8a6628ea46b6f6882f9cde34278e7091ce538bb0fa31620ccac79179239f0f0e: Status 404 returned error can't find the container with id 8a6628ea46b6f6882f9cde34278e7091ce538bb0fa31620ccac79179239f0f0e Jul 27 21:26:43.859061 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.859015 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:26:43.859141 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.859113 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:26:43.859179 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:43.859161 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:26:44.271146 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:44.271100 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" event={"ID":"267f2d48-d276-4d69-a9ae-053326ef42f6","Type":"ContainerStarted","Data":"72cc623f36c69b2c9cc14d5282a0d68ff261b4f9a6ba51528b8fe06224c56593"} Jul 27 21:26:44.271367 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:44.271171 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" event={"ID":"267f2d48-d276-4d69-a9ae-053326ef42f6","Type":"ContainerStarted","Data":"8a6628ea46b6f6882f9cde34278e7091ce538bb0fa31620ccac79179239f0f0e"} Jul 27 21:26:44.295958 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:44.295908 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" podStartSLOduration=1.295889823 podStartE2EDuration="1.295889823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:26:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:26:44.295161688 +0000 UTC m=+2132.818279423" watchObservedRunningTime="2026-07-27 21:26:44.295889823 +0000 UTC m=+2132.819007622" Jul 27 21:26:44.687271 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:44.687193 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:44.692667 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:44.692639 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:45.280255 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:45.280194 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:45.285394 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:45.285363 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:26:50.309268 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:50.309168 2565 generic.go:356] "Generic (PLEG): container finished" podID="dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" containerID="7e4aca0ed67b6a960ac1c7f045eee55185b07e82c4a502eff603dacccdef1182" exitCode=0 Jul 27 21:26:50.309639 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:50.309264 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" event={"ID":"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2","Type":"ContainerDied","Data":"7e4aca0ed67b6a960ac1c7f045eee55185b07e82c4a502eff603dacccdef1182"} Jul 27 21:26:50.309639 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:50.309634 2565 scope.go:117] "RemoveContainer" containerID="7e4aca0ed67b6a960ac1c7f045eee55185b07e82c4a502eff603dacccdef1182" Jul 27 21:26:50.310764 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:50.310738 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:26:50.310830 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:50.310810 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:26:50.310889 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:50.310856 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:26:51.316026 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:51.315992 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" event={"ID":"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2","Type":"ContainerStarted","Data":"e8dfe2062aa89dea3f692e1e4a8fdcb78bc6121ed0d61cef85ae07139b3ec28e"} Jul 27 21:26:51.528943 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:51.528851 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:51.529998 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:51.529971 2565 patch_prober.go:28] interesting pod/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 27 21:26:51.530119 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:51.530021 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" podUID="dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 27 21:26:52.225707 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:52.225672 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:52.528106 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:52.528006 2565 patch_prober.go:28] interesting pod/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 27 21:26:52.528106 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:52.528063 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" podUID="dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 27 21:26:53.529092 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:53.529053 2565 patch_prober.go:28] interesting pod/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 27 21:26:53.529508 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:53.529127 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" podUID="dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 27 21:26:54.528758 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:54.528714 2565 patch_prober.go:28] interesting pod/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 27 21:26:54.528994 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:54.528771 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" podUID="dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 27 21:26:55.532312 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:55.532279 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:26:55.533546 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:26:55.533523 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:27:01.321454 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:01.321411 2565 patch_prober.go:28] interesting pod/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 27 21:27:01.322982 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:01.321487 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" podUID="267f2d48-d276-4d69-a9ae-053326ef42f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 27 21:27:02.376154 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:02.376121 2565 generic.go:356] "Generic (PLEG): container finished" podID="267f2d48-d276-4d69-a9ae-053326ef42f6" containerID="72cc623f36c69b2c9cc14d5282a0d68ff261b4f9a6ba51528b8fe06224c56593" exitCode=0 Jul 27 21:27:02.376560 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:02.376195 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" event={"ID":"267f2d48-d276-4d69-a9ae-053326ef42f6","Type":"ContainerDied","Data":"72cc623f36c69b2c9cc14d5282a0d68ff261b4f9a6ba51528b8fe06224c56593"} Jul 27 21:27:02.376640 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:02.376624 2565 scope.go:117] "RemoveContainer" containerID="72cc623f36c69b2c9cc14d5282a0d68ff261b4f9a6ba51528b8fe06224c56593" Jul 27 21:27:02.377748 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:02.377718 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:27:02.377827 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:02.377785 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:27:02.377827 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:02.377818 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:27:03.387970 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:03.387923 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" event={"ID":"267f2d48-d276-4d69-a9ae-053326ef42f6","Type":"ContainerStarted","Data":"48c4e18be8ed65567e2ecb179217a5310b478a4264c35d61db72a26e6e7b088c"} Jul 27 21:27:03.687080 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:03.687041 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:27:03.687444 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:03.687414 2565 patch_prober.go:28] interesting pod/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 27 21:27:03.687538 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:03.687459 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" podUID="267f2d48-d276-4d69-a9ae-053326ef42f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 27 21:27:04.687301 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:04.687262 2565 patch_prober.go:28] interesting pod/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 27 21:27:04.687687 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:04.687327 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" podUID="267f2d48-d276-4d69-a9ae-053326ef42f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 27 21:27:05.687540 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:05.687499 2565 patch_prober.go:28] interesting pod/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 27 21:27:05.688309 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:05.687566 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" podUID="267f2d48-d276-4d69-a9ae-053326ef42f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 27 21:27:06.687188 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:06.687141 2565 patch_prober.go:28] interesting pod/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 27 21:27:06.687416 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:06.687229 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" podUID="267f2d48-d276-4d69-a9ae-053326ef42f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 27 21:27:07.689865 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:07.689832 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:27:07.690310 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:07.690106 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:27:07.690906 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:07.690887 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:27:33.683548 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.683507 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg"] Jul 27 21:27:33.686442 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.686425 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:33.691793 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.691762 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0adc64\"/\"test-model-0adc64-kserve-self-signed-certs\"" Jul 27 21:27:33.691925 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.691831 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0adc64\"/\"default-dockercfg-pwdjm\"" Jul 27 21:27:33.692342 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.692326 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0adc64\"/\"openshift-service-ca.crt\"" Jul 27 21:27:33.693279 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.693262 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0adc64\"/\"kube-root-ca.crt\"" Jul 27 21:27:33.723366 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.723325 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg"] Jul 27 21:27:33.765510 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.765472 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-home\") pod \"test-model-0adc64-kserve-6d7f7b9df-2c4zg\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:33.765680 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.765518 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b284365-64ca-4a16-b6a6-461c34b486fa-tls-certs\") pod \"test-model-0adc64-kserve-6d7f7b9df-2c4zg\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:33.765680 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.765546 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-dshm\") pod \"test-model-0adc64-kserve-6d7f7b9df-2c4zg\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:33.765680 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.765607 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-model-cache\") pod \"test-model-0adc64-kserve-6d7f7b9df-2c4zg\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:33.765680 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.765654 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-tmp-dir\") pod \"test-model-0adc64-kserve-6d7f7b9df-2c4zg\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:33.765818 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.765738 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbnh5\" (UniqueName: \"kubernetes.io/projected/9b284365-64ca-4a16-b6a6-461c34b486fa-kube-api-access-wbnh5\") pod \"test-model-0adc64-kserve-6d7f7b9df-2c4zg\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:33.866280 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.866247 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-model-cache\") pod \"test-model-0adc64-kserve-6d7f7b9df-2c4zg\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:33.866472 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.866300 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-tmp-dir\") pod \"test-model-0adc64-kserve-6d7f7b9df-2c4zg\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:33.866472 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.866385 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbnh5\" (UniqueName: \"kubernetes.io/projected/9b284365-64ca-4a16-b6a6-461c34b486fa-kube-api-access-wbnh5\") pod \"test-model-0adc64-kserve-6d7f7b9df-2c4zg\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:33.866472 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.866422 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-home\") pod \"test-model-0adc64-kserve-6d7f7b9df-2c4zg\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:33.866472 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.866456 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b284365-64ca-4a16-b6a6-461c34b486fa-tls-certs\") pod \"test-model-0adc64-kserve-6d7f7b9df-2c4zg\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:33.866691 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.866486 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-dshm\") pod \"test-model-0adc64-kserve-6d7f7b9df-2c4zg\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:33.866691 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.866633 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-model-cache\") pod \"test-model-0adc64-kserve-6d7f7b9df-2c4zg\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:33.866691 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.866645 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-tmp-dir\") pod \"test-model-0adc64-kserve-6d7f7b9df-2c4zg\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:33.866852 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.866779 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-home\") pod \"test-model-0adc64-kserve-6d7f7b9df-2c4zg\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:33.868852 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.868817 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-dshm\") pod \"test-model-0adc64-kserve-6d7f7b9df-2c4zg\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:33.869104 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.869087 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b284365-64ca-4a16-b6a6-461c34b486fa-tls-certs\") pod \"test-model-0adc64-kserve-6d7f7b9df-2c4zg\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:33.879523 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.879499 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbnh5\" (UniqueName: \"kubernetes.io/projected/9b284365-64ca-4a16-b6a6-461c34b486fa-kube-api-access-wbnh5\") pod \"test-model-0adc64-kserve-6d7f7b9df-2c4zg\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:33.995393 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:33.995296 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:34.149768 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:27:34.149737 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b284365_64ca_4a16_b6a6_461c34b486fa.slice/crio-f93ff513dd1b7e8c07cce0721db3de2b5ba22104396cd50fcb597c38c8a323da WatchSource:0}: Error finding container f93ff513dd1b7e8c07cce0721db3de2b5ba22104396cd50fcb597c38c8a323da: Status 404 returned error can't find the container with id f93ff513dd1b7e8c07cce0721db3de2b5ba22104396cd50fcb597c38c8a323da Jul 27 21:27:34.150700 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:34.150676 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg"] Jul 27 21:27:34.544736 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:34.544703 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" event={"ID":"9b284365-64ca-4a16-b6a6-461c34b486fa","Type":"ContainerStarted","Data":"51a8235c74edf44abfb2e2952634dc82c49916e2c3317b7d197e2b5d1dceb3b2"} Jul 27 21:27:34.544916 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:34.544750 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" event={"ID":"9b284365-64ca-4a16-b6a6-461c34b486fa","Type":"ContainerStarted","Data":"f93ff513dd1b7e8c07cce0721db3de2b5ba22104396cd50fcb597c38c8a323da"} Jul 27 21:27:34.579161 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:34.579103 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" podStartSLOduration=1.579086437 podStartE2EDuration="1.579086437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:27:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:27:34.578340577 +0000 UTC m=+2183.101458299" watchObservedRunningTime="2026-07-27 21:27:34.579086437 +0000 UTC m=+2183.102204159" Jul 27 21:27:43.996048 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:43.995994 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:43.996488 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:43.996068 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:44.008518 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:44.008494 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:27:44.606023 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:27:44.605995 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:28:21.871505 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:21.871469 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg"] Jul 27 21:28:36.890741 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:36.890657 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" podUID="9b284365-64ca-4a16-b6a6-461c34b486fa" containerName="main" containerID="cri-o://51a8235c74edf44abfb2e2952634dc82c49916e2c3317b7d197e2b5d1dceb3b2" gracePeriod=45 Jul 27 21:28:37.143306 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.143246 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:28:37.247689 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.247659 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbnh5\" (UniqueName: \"kubernetes.io/projected/9b284365-64ca-4a16-b6a6-461c34b486fa-kube-api-access-wbnh5\") pod \"9b284365-64ca-4a16-b6a6-461c34b486fa\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " Jul 27 21:28:37.247870 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.247709 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-home\") pod \"9b284365-64ca-4a16-b6a6-461c34b486fa\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " Jul 27 21:28:37.247870 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.247728 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-tmp-dir\") pod \"9b284365-64ca-4a16-b6a6-461c34b486fa\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " Jul 27 21:28:37.247870 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.247778 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b284365-64ca-4a16-b6a6-461c34b486fa-tls-certs\") pod \"9b284365-64ca-4a16-b6a6-461c34b486fa\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " Jul 27 21:28:37.247870 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.247795 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-dshm\") pod \"9b284365-64ca-4a16-b6a6-461c34b486fa\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " Jul 27 21:28:37.247870 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.247821 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-model-cache\") pod \"9b284365-64ca-4a16-b6a6-461c34b486fa\" (UID: \"9b284365-64ca-4a16-b6a6-461c34b486fa\") " Jul 27 21:28:37.248136 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.247995 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-home" (OuterVolumeSpecName: "home") pod "9b284365-64ca-4a16-b6a6-461c34b486fa" (UID: "9b284365-64ca-4a16-b6a6-461c34b486fa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:28:37.248136 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.248028 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9b284365-64ca-4a16-b6a6-461c34b486fa" (UID: "9b284365-64ca-4a16-b6a6-461c34b486fa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:28:37.248136 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.248122 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-home\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:28:37.248295 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.248141 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-tmp-dir\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:28:37.248295 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.248158 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-model-cache" (OuterVolumeSpecName: "model-cache") pod "9b284365-64ca-4a16-b6a6-461c34b486fa" (UID: "9b284365-64ca-4a16-b6a6-461c34b486fa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:28:37.250040 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.250016 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-dshm" (OuterVolumeSpecName: "dshm") pod "9b284365-64ca-4a16-b6a6-461c34b486fa" (UID: "9b284365-64ca-4a16-b6a6-461c34b486fa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:28:37.250150 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.250133 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b284365-64ca-4a16-b6a6-461c34b486fa-kube-api-access-wbnh5" (OuterVolumeSpecName: "kube-api-access-wbnh5") pod "9b284365-64ca-4a16-b6a6-461c34b486fa" (UID: "9b284365-64ca-4a16-b6a6-461c34b486fa"). InnerVolumeSpecName "kube-api-access-wbnh5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:28:37.250199 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.250150 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b284365-64ca-4a16-b6a6-461c34b486fa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9b284365-64ca-4a16-b6a6-461c34b486fa" (UID: "9b284365-64ca-4a16-b6a6-461c34b486fa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 21:28:37.354641 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.349485 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b284365-64ca-4a16-b6a6-461c34b486fa-tls-certs\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:28:37.354641 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.349513 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-dshm\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:28:37.354641 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.349525 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b284365-64ca-4a16-b6a6-461c34b486fa-model-cache\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:28:37.354641 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.349547 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbnh5\" (UniqueName: \"kubernetes.io/projected/9b284365-64ca-4a16-b6a6-461c34b486fa-kube-api-access-wbnh5\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:28:37.864480 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.864441 2565 generic.go:356] "Generic (PLEG): container finished" podID="9b284365-64ca-4a16-b6a6-461c34b486fa" containerID="51a8235c74edf44abfb2e2952634dc82c49916e2c3317b7d197e2b5d1dceb3b2" exitCode=0 Jul 27 21:28:37.864674 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.864502 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" Jul 27 21:28:37.864674 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.864534 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" event={"ID":"9b284365-64ca-4a16-b6a6-461c34b486fa","Type":"ContainerDied","Data":"51a8235c74edf44abfb2e2952634dc82c49916e2c3317b7d197e2b5d1dceb3b2"} Jul 27 21:28:37.864674 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.864578 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg" event={"ID":"9b284365-64ca-4a16-b6a6-461c34b486fa","Type":"ContainerDied","Data":"f93ff513dd1b7e8c07cce0721db3de2b5ba22104396cd50fcb597c38c8a323da"} Jul 27 21:28:37.864674 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.864594 2565 scope.go:117] "RemoveContainer" containerID="51a8235c74edf44abfb2e2952634dc82c49916e2c3317b7d197e2b5d1dceb3b2" Jul 27 21:28:37.874470 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.874453 2565 scope.go:117] "RemoveContainer" containerID="51a8235c74edf44abfb2e2952634dc82c49916e2c3317b7d197e2b5d1dceb3b2" Jul 27 21:28:37.874731 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:28:37.874712 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51a8235c74edf44abfb2e2952634dc82c49916e2c3317b7d197e2b5d1dceb3b2\": container with ID starting with 51a8235c74edf44abfb2e2952634dc82c49916e2c3317b7d197e2b5d1dceb3b2 not found: ID does not exist" containerID="51a8235c74edf44abfb2e2952634dc82c49916e2c3317b7d197e2b5d1dceb3b2" Jul 27 21:28:37.874788 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.874742 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51a8235c74edf44abfb2e2952634dc82c49916e2c3317b7d197e2b5d1dceb3b2"} err="failed to get container status \"51a8235c74edf44abfb2e2952634dc82c49916e2c3317b7d197e2b5d1dceb3b2\": rpc error: code = NotFound desc = could not find container \"51a8235c74edf44abfb2e2952634dc82c49916e2c3317b7d197e2b5d1dceb3b2\": container with ID starting with 51a8235c74edf44abfb2e2952634dc82c49916e2c3317b7d197e2b5d1dceb3b2 not found: ID does not exist" Jul 27 21:28:37.897734 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.897700 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg"] Jul 27 21:28:37.922596 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:37.922566 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-0adc64/test-model-0adc64-kserve-6d7f7b9df-2c4zg"] Jul 27 21:28:38.052588 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:38.052557 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b284365-64ca-4a16-b6a6-461c34b486fa" path="/var/lib/kubelet/pods/9b284365-64ca-4a16-b6a6-461c34b486fa/volumes" Jul 27 21:28:44.726455 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:44.726422 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv"] Jul 27 21:28:44.726933 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:44.726668 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" podUID="dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" containerName="istio-proxy" containerID="cri-o://e8dfe2062aa89dea3f692e1e4a8fdcb78bc6121ed0d61cef85ae07139b3ec28e" gracePeriod=30 Jul 27 21:28:49.928664 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:49.928631 2565 generic.go:356] "Generic (PLEG): container finished" podID="dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" containerID="e8dfe2062aa89dea3f692e1e4a8fdcb78bc6121ed0d61cef85ae07139b3ec28e" exitCode=0 Jul 27 21:28:49.929078 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:49.928709 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" event={"ID":"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2","Type":"ContainerDied","Data":"e8dfe2062aa89dea3f692e1e4a8fdcb78bc6121ed0d61cef85ae07139b3ec28e"} Jul 27 21:28:49.929078 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:49.928833 2565 scope.go:117] "RemoveContainer" containerID="7e4aca0ed67b6a960ac1c7f045eee55185b07e82c4a502eff603dacccdef1182" Jul 27 21:28:49.969222 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:49.969201 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:28:50.057354 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.057283 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-envoy\") pod \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " Jul 27 21:28:50.057354 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.057311 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istiod-ca-cert\") pod \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " Jul 27 21:28:50.057354 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.057341 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-podinfo\") pod \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " Jul 27 21:28:50.057652 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.057371 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-workload-certs\") pod \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " Jul 27 21:28:50.057652 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.057412 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-workload-socket\") pod \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " Jul 27 21:28:50.057652 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.057435 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msjq5\" (UniqueName: \"kubernetes.io/projected/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-kube-api-access-msjq5\") pod \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " Jul 27 21:28:50.057652 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.057452 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-data\") pod \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " Jul 27 21:28:50.057652 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.057480 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-token\") pod \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " Jul 27 21:28:50.057652 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.057507 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-credential-socket\") pod \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\" (UID: \"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2\") " Jul 27 21:28:50.057967 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.057763 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" (UID: "dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:28:50.057967 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.057793 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" (UID: "dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 21:28:50.057967 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.057929 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" (UID: "dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:28:50.057967 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.057945 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" (UID: "dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:28:50.058704 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.058672 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" (UID: "dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:28:50.059770 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.059739 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" (UID: "dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:28:50.059865 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.059767 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" (UID: "dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 27 21:28:50.059865 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.059843 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" (UID: "dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:28:50.059941 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.059854 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-kube-api-access-msjq5" (OuterVolumeSpecName: "kube-api-access-msjq5") pod "dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" (UID: "dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2"). InnerVolumeSpecName "kube-api-access-msjq5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:28:50.158088 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.158064 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-workload-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:28:50.158088 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.158084 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msjq5\" (UniqueName: \"kubernetes.io/projected/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-kube-api-access-msjq5\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:28:50.158224 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.158094 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-data\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:28:50.158224 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.158103 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-token\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:28:50.158224 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.158111 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-credential-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:28:50.158224 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.158119 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-envoy\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:28:50.158224 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.158127 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istiod-ca-cert\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:28:50.158224 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.158134 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-istio-podinfo\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:28:50.158224 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.158141 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2-workload-certs\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:28:50.935124 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.935091 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" Jul 27 21:28:50.935574 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.935094 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv" event={"ID":"dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2","Type":"ContainerDied","Data":"4cb59414f2cab4f2f5b3975fb0e15120739a2d957964d7b22b7f08c687303f5e"} Jul 27 21:28:50.935574 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.935228 2565 scope.go:117] "RemoveContainer" containerID="e8dfe2062aa89dea3f692e1e4a8fdcb78bc6121ed0d61cef85ae07139b3ec28e" Jul 27 21:28:50.962907 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.962874 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv"] Jul 27 21:28:50.965118 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:50.965092 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-edfab0-openshift-default-5d4894cb6-m9rqv"] Jul 27 21:28:52.057721 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:28:52.057686 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" path="/var/lib/kubelet/pods/dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2/volumes" Jul 27 21:29:08.296806 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:08.296770 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx"] Jul 27 21:29:08.297175 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:08.297008 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" podUID="267f2d48-d276-4d69-a9ae-053326ef42f6" containerName="istio-proxy" containerID="cri-o://48c4e18be8ed65567e2ecb179217a5310b478a4264c35d61db72a26e6e7b088c" gracePeriod=30 Jul 27 21:29:09.340454 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.340420 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9"] Jul 27 21:29:09.340822 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.340800 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" containerName="istio-proxy" Jul 27 21:29:09.340822 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.340811 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" containerName="istio-proxy" Jul 27 21:29:09.340822 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.340820 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" containerName="istio-proxy" Jul 27 21:29:09.340931 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.340826 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" containerName="istio-proxy" Jul 27 21:29:09.340931 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.340856 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b284365-64ca-4a16-b6a6-461c34b486fa" containerName="main" Jul 27 21:29:09.340931 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.340862 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b284365-64ca-4a16-b6a6-461c34b486fa" containerName="main" Jul 27 21:29:09.340931 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.340919 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b284365-64ca-4a16-b6a6-461c34b486fa" containerName="main" Jul 27 21:29:09.340931 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.340929 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" containerName="istio-proxy" Jul 27 21:29:09.341082 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.340935 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc06b3fe-69d1-4311-ade0-ee4e0dbdb8e2" containerName="istio-proxy" Jul 27 21:29:09.344002 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.343983 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.351318 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.351290 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-c3ee91-openshift-default-dockercfg-m5scs\"" Jul 27 21:29:09.373607 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.373584 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9"] Jul 27 21:29:09.422108 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.422071 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-workload-certs\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.422361 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.422133 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7856196d-a32c-400c-b11d-eff4d8c24fb6-istiod-ca-cert\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.422361 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.422279 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-envoy\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.422497 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.422375 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-credential-socket\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.422497 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.422434 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-data\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.422497 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.422462 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-podinfo\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.422497 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.422489 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-workload-socket\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.423003 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.422536 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7gtf\" (UniqueName: \"kubernetes.io/projected/7856196d-a32c-400c-b11d-eff4d8c24fb6-kube-api-access-h7gtf\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.423003 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.422610 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-token\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.523974 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.523929 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-token\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.524175 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.523996 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-workload-certs\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.524175 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.524029 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7856196d-a32c-400c-b11d-eff4d8c24fb6-istiod-ca-cert\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.524175 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.524060 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-envoy\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.524175 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.524085 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-credential-socket\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.524175 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.524124 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-data\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.524175 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.524148 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-podinfo\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.524175 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.524171 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-workload-socket\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.524573 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.524212 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7gtf\" (UniqueName: \"kubernetes.io/projected/7856196d-a32c-400c-b11d-eff4d8c24fb6-kube-api-access-h7gtf\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.524573 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.524451 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-workload-certs\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.524677 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.524646 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-data\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.524738 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.524691 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-workload-socket\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.524935 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.524914 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-credential-socket\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.525017 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.524995 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7856196d-a32c-400c-b11d-eff4d8c24fb6-istiod-ca-cert\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.526469 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.526446 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-envoy\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.526764 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.526746 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-podinfo\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.534996 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.534977 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-token\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.535108 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.535089 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7gtf\" (UniqueName: \"kubernetes.io/projected/7856196d-a32c-400c-b11d-eff4d8c24fb6-kube-api-access-h7gtf\") pod \"e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.659130 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.659046 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:09.818695 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.818667 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9"] Jul 27 21:29:09.819976 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:29:09.819951 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7856196d_a32c_400c_b11d_eff4d8c24fb6.slice/crio-826f478f7044db95123a97e594a889cb79c2efa53b8b16b8ec12ea4d00940840 WatchSource:0}: Error finding container 826f478f7044db95123a97e594a889cb79c2efa53b8b16b8ec12ea4d00940840: Status 404 returned error can't find the container with id 826f478f7044db95123a97e594a889cb79c2efa53b8b16b8ec12ea4d00940840 Jul 27 21:29:09.822104 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.822071 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:29:09.822206 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.822156 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:29:09.822303 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:09.822202 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:29:10.031111 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:10.031064 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" event={"ID":"7856196d-a32c-400c-b11d-eff4d8c24fb6","Type":"ContainerStarted","Data":"e74a41f05645741ff1e50bddc623e54e44e904fdab710021feba6bd7d09ae73c"} Jul 27 21:29:10.031336 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:10.031138 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" event={"ID":"7856196d-a32c-400c-b11d-eff4d8c24fb6","Type":"ContainerStarted","Data":"826f478f7044db95123a97e594a889cb79c2efa53b8b16b8ec12ea4d00940840"} Jul 27 21:29:10.076396 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:10.076349 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" podStartSLOduration=1.076331941 podStartE2EDuration="1.076331941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:29:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:29:10.074819869 +0000 UTC m=+2278.597937591" watchObservedRunningTime="2026-07-27 21:29:10.076331941 +0000 UTC m=+2278.599449663" Jul 27 21:29:10.659461 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:10.659426 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:10.664329 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:10.664301 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:11.037386 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:11.037353 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:11.038454 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:11.038427 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:13.557586 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.557528 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:29:13.661891 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.661856 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-workload-socket\") pod \"267f2d48-d276-4d69-a9ae-053326ef42f6\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " Jul 27 21:29:13.662068 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.661906 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/267f2d48-d276-4d69-a9ae-053326ef42f6-istiod-ca-cert\") pod \"267f2d48-d276-4d69-a9ae-053326ef42f6\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " Jul 27 21:29:13.662068 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.661947 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-data\") pod \"267f2d48-d276-4d69-a9ae-053326ef42f6\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " Jul 27 21:29:13.662068 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.661973 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-podinfo\") pod \"267f2d48-d276-4d69-a9ae-053326ef42f6\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " Jul 27 21:29:13.662068 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.661997 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-token\") pod \"267f2d48-d276-4d69-a9ae-053326ef42f6\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " Jul 27 21:29:13.662068 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.662044 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcd7m\" (UniqueName: \"kubernetes.io/projected/267f2d48-d276-4d69-a9ae-053326ef42f6-kube-api-access-dcd7m\") pod \"267f2d48-d276-4d69-a9ae-053326ef42f6\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " Jul 27 21:29:13.662373 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.662081 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-workload-certs\") pod \"267f2d48-d276-4d69-a9ae-053326ef42f6\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " Jul 27 21:29:13.662373 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.662172 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-credential-socket\") pod \"267f2d48-d276-4d69-a9ae-053326ef42f6\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " Jul 27 21:29:13.662373 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.662206 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-envoy\") pod \"267f2d48-d276-4d69-a9ae-053326ef42f6\" (UID: \"267f2d48-d276-4d69-a9ae-053326ef42f6\") " Jul 27 21:29:13.662373 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.662266 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "267f2d48-d276-4d69-a9ae-053326ef42f6" (UID: "267f2d48-d276-4d69-a9ae-053326ef42f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:29:13.662573 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.662535 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-workload-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:29:13.663478 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.663100 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-data" (OuterVolumeSpecName: "istio-data") pod "267f2d48-d276-4d69-a9ae-053326ef42f6" (UID: "267f2d48-d276-4d69-a9ae-053326ef42f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:29:13.663478 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.663337 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "267f2d48-d276-4d69-a9ae-053326ef42f6" (UID: "267f2d48-d276-4d69-a9ae-053326ef42f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:29:13.665737 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.663494 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/267f2d48-d276-4d69-a9ae-053326ef42f6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "267f2d48-d276-4d69-a9ae-053326ef42f6" (UID: "267f2d48-d276-4d69-a9ae-053326ef42f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 21:29:13.665737 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.663082 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "267f2d48-d276-4d69-a9ae-053326ef42f6" (UID: "267f2d48-d276-4d69-a9ae-053326ef42f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:29:13.665737 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.665386 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "267f2d48-d276-4d69-a9ae-053326ef42f6" (UID: "267f2d48-d276-4d69-a9ae-053326ef42f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:29:13.666024 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.665853 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/267f2d48-d276-4d69-a9ae-053326ef42f6-kube-api-access-dcd7m" (OuterVolumeSpecName: "kube-api-access-dcd7m") pod "267f2d48-d276-4d69-a9ae-053326ef42f6" (UID: "267f2d48-d276-4d69-a9ae-053326ef42f6"). InnerVolumeSpecName "kube-api-access-dcd7m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:29:13.666510 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.666484 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "267f2d48-d276-4d69-a9ae-053326ef42f6" (UID: "267f2d48-d276-4d69-a9ae-053326ef42f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 27 21:29:13.668067 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.668036 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-token" (OuterVolumeSpecName: "istio-token") pod "267f2d48-d276-4d69-a9ae-053326ef42f6" (UID: "267f2d48-d276-4d69-a9ae-053326ef42f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:29:13.763989 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.763951 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcd7m\" (UniqueName: \"kubernetes.io/projected/267f2d48-d276-4d69-a9ae-053326ef42f6-kube-api-access-dcd7m\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:29:13.763989 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.763981 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-workload-certs\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:29:13.763989 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.763991 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-credential-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:29:13.763989 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.763999 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-envoy\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:29:13.764412 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.764008 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/267f2d48-d276-4d69-a9ae-053326ef42f6-istiod-ca-cert\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:29:13.764412 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.764016 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-data\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:29:13.764412 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.764024 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-podinfo\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:29:13.764412 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:13.764031 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/267f2d48-d276-4d69-a9ae-053326ef42f6-istio-token\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:29:14.058717 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:14.058686 2565 generic.go:356] "Generic (PLEG): container finished" podID="267f2d48-d276-4d69-a9ae-053326ef42f6" containerID="48c4e18be8ed65567e2ecb179217a5310b478a4264c35d61db72a26e6e7b088c" exitCode=0 Jul 27 21:29:14.058880 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:14.058769 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" Jul 27 21:29:14.058880 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:14.058800 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" event={"ID":"267f2d48-d276-4d69-a9ae-053326ef42f6","Type":"ContainerDied","Data":"48c4e18be8ed65567e2ecb179217a5310b478a4264c35d61db72a26e6e7b088c"} Jul 27 21:29:14.058880 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:14.058854 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx" event={"ID":"267f2d48-d276-4d69-a9ae-053326ef42f6","Type":"ContainerDied","Data":"8a6628ea46b6f6882f9cde34278e7091ce538bb0fa31620ccac79179239f0f0e"} Jul 27 21:29:14.058880 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:14.058877 2565 scope.go:117] "RemoveContainer" containerID="48c4e18be8ed65567e2ecb179217a5310b478a4264c35d61db72a26e6e7b088c" Jul 27 21:29:14.069422 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:14.069405 2565 scope.go:117] "RemoveContainer" containerID="72cc623f36c69b2c9cc14d5282a0d68ff261b4f9a6ba51528b8fe06224c56593" Jul 27 21:29:14.077031 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:14.077006 2565 scope.go:117] "RemoveContainer" containerID="48c4e18be8ed65567e2ecb179217a5310b478a4264c35d61db72a26e6e7b088c" Jul 27 21:29:14.077389 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:29:14.077368 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48c4e18be8ed65567e2ecb179217a5310b478a4264c35d61db72a26e6e7b088c\": container with ID starting with 48c4e18be8ed65567e2ecb179217a5310b478a4264c35d61db72a26e6e7b088c not found: ID does not exist" containerID="48c4e18be8ed65567e2ecb179217a5310b478a4264c35d61db72a26e6e7b088c" Jul 27 21:29:14.077448 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:14.077397 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48c4e18be8ed65567e2ecb179217a5310b478a4264c35d61db72a26e6e7b088c"} err="failed to get container status \"48c4e18be8ed65567e2ecb179217a5310b478a4264c35d61db72a26e6e7b088c\": rpc error: code = NotFound desc = could not find container \"48c4e18be8ed65567e2ecb179217a5310b478a4264c35d61db72a26e6e7b088c\": container with ID starting with 48c4e18be8ed65567e2ecb179217a5310b478a4264c35d61db72a26e6e7b088c not found: ID does not exist" Jul 27 21:29:14.077448 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:14.077416 2565 scope.go:117] "RemoveContainer" containerID="72cc623f36c69b2c9cc14d5282a0d68ff261b4f9a6ba51528b8fe06224c56593" Jul 27 21:29:14.077649 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:29:14.077633 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72cc623f36c69b2c9cc14d5282a0d68ff261b4f9a6ba51528b8fe06224c56593\": container with ID starting with 72cc623f36c69b2c9cc14d5282a0d68ff261b4f9a6ba51528b8fe06224c56593 not found: ID does not exist" containerID="72cc623f36c69b2c9cc14d5282a0d68ff261b4f9a6ba51528b8fe06224c56593" Jul 27 21:29:14.077712 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:14.077652 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72cc623f36c69b2c9cc14d5282a0d68ff261b4f9a6ba51528b8fe06224c56593"} err="failed to get container status \"72cc623f36c69b2c9cc14d5282a0d68ff261b4f9a6ba51528b8fe06224c56593\": rpc error: code = NotFound desc = could not find container \"72cc623f36c69b2c9cc14d5282a0d68ff261b4f9a6ba51528b8fe06224c56593\": container with ID starting with 72cc623f36c69b2c9cc14d5282a0d68ff261b4f9a6ba51528b8fe06224c56593 not found: ID does not exist" Jul 27 21:29:14.121070 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:14.121045 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx"] Jul 27 21:29:14.127617 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:14.127597 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-0adc64-openshift-default-5c694997c6-hj4jx"] Jul 27 21:29:16.053535 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:16.053500 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="267f2d48-d276-4d69-a9ae-053326ef42f6" path="/var/lib/kubelet/pods/267f2d48-d276-4d69-a9ae-053326ef42f6/volumes" Jul 27 21:29:17.503666 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.503634 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt"] Jul 27 21:29:17.504032 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.503992 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="267f2d48-d276-4d69-a9ae-053326ef42f6" containerName="istio-proxy" Jul 27 21:29:17.504032 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.504002 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="267f2d48-d276-4d69-a9ae-053326ef42f6" containerName="istio-proxy" Jul 27 21:29:17.504032 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.504017 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="267f2d48-d276-4d69-a9ae-053326ef42f6" containerName="istio-proxy" Jul 27 21:29:17.504032 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.504022 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="267f2d48-d276-4d69-a9ae-053326ef42f6" containerName="istio-proxy" Jul 27 21:29:17.504170 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.504078 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="267f2d48-d276-4d69-a9ae-053326ef42f6" containerName="istio-proxy" Jul 27 21:29:17.504170 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.504087 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="267f2d48-d276-4d69-a9ae-053326ef42f6" containerName="istio-proxy" Jul 27 21:29:17.506737 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.506720 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.510918 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.510892 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-86cc77-openshift-default-dockercfg-wwjs8\"" Jul 27 21:29:17.543335 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.543307 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt"] Jul 27 21:29:17.599039 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.599004 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-podinfo\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.599208 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.599046 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-token\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.599208 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.599079 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-credential-socket\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.599208 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.599134 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-data\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.599208 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.599168 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-workload-certs\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.599208 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.599186 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8hm9\" (UniqueName: \"kubernetes.io/projected/f672c708-8e8c-4489-b3a7-25c4e85be824-kube-api-access-s8hm9\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.599208 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.599206 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f672c708-8e8c-4489-b3a7-25c4e85be824-istiod-ca-cert\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.599564 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.599336 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-envoy\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.599564 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.599384 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-workload-socket\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.700620 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.700583 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-credential-socket\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.700812 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.700640 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-data\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.700812 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.700675 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-workload-certs\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.700812 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.700700 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8hm9\" (UniqueName: \"kubernetes.io/projected/f672c708-8e8c-4489-b3a7-25c4e85be824-kube-api-access-s8hm9\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.700812 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.700724 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f672c708-8e8c-4489-b3a7-25c4e85be824-istiod-ca-cert\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.701045 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.700862 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-envoy\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.701045 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.700925 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-workload-socket\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.701045 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.700995 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-podinfo\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.701045 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.701006 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-data\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.701631 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.701413 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-token\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.701631 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.701497 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-workload-certs\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.701631 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.701051 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-credential-socket\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.701631 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.701583 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-workload-socket\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.701904 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.701822 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f672c708-8e8c-4489-b3a7-25c4e85be824-istiod-ca-cert\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.709928 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.709748 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-envoy\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.711578 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.711559 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-podinfo\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.718197 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.718179 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8hm9\" (UniqueName: \"kubernetes.io/projected/f672c708-8e8c-4489-b3a7-25c4e85be824-kube-api-access-s8hm9\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.718369 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.718350 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-token\") pod \"e2e-api-b-86cc77-openshift-default-79cf45f494-297tt\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.817226 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.817134 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:17.995807 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:17.995700 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt"] Jul 27 21:29:17.998200 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:29:17.998169 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf672c708_8e8c_4489_b3a7_25c4e85be824.slice/crio-c2e8f19e2e9481f871bb47c57e106e0f97efe28299f20b9dcaa18b6b2929c9fa WatchSource:0}: Error finding container c2e8f19e2e9481f871bb47c57e106e0f97efe28299f20b9dcaa18b6b2929c9fa: Status 404 returned error can't find the container with id c2e8f19e2e9481f871bb47c57e106e0f97efe28299f20b9dcaa18b6b2929c9fa Jul 27 21:29:18.000297 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:18.000263 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:29:18.000387 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:18.000332 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:29:18.000387 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:18.000359 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:29:18.094908 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:18.094816 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" event={"ID":"f672c708-8e8c-4489-b3a7-25c4e85be824","Type":"ContainerStarted","Data":"df7b892aea3343579095db64f93861420a1ef6a6e746968935b2216cf5ab5f14"} Jul 27 21:29:18.094908 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:18.094873 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" event={"ID":"f672c708-8e8c-4489-b3a7-25c4e85be824","Type":"ContainerStarted","Data":"c2e8f19e2e9481f871bb47c57e106e0f97efe28299f20b9dcaa18b6b2929c9fa"} Jul 27 21:29:18.153503 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:18.153437 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" podStartSLOduration=1.153415715 podStartE2EDuration="1.153415715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:29:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:29:18.152273441 +0000 UTC m=+2286.675391162" watchObservedRunningTime="2026-07-27 21:29:18.153415715 +0000 UTC m=+2286.676533438" Jul 27 21:29:18.818074 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:18.818034 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:19.822587 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:19.822558 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:20.104775 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:20.104677 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:20.105644 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:20.105623 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:24.130536 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:24.130450 2565 generic.go:356] "Generic (PLEG): container finished" podID="7856196d-a32c-400c-b11d-eff4d8c24fb6" containerID="e74a41f05645741ff1e50bddc623e54e44e904fdab710021feba6bd7d09ae73c" exitCode=0 Jul 27 21:29:24.130948 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:24.130530 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" event={"ID":"7856196d-a32c-400c-b11d-eff4d8c24fb6","Type":"ContainerDied","Data":"e74a41f05645741ff1e50bddc623e54e44e904fdab710021feba6bd7d09ae73c"} Jul 27 21:29:24.130948 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:24.130939 2565 scope.go:117] "RemoveContainer" containerID="e74a41f05645741ff1e50bddc623e54e44e904fdab710021feba6bd7d09ae73c" Jul 27 21:29:24.131940 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:24.131910 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:29:24.132038 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:24.131994 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:29:24.132103 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:24.132045 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:29:25.137102 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:25.137067 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" event={"ID":"7856196d-a32c-400c-b11d-eff4d8c24fb6","Type":"ContainerStarted","Data":"3d082289801b61dfb60c521d437a04ced876a75bcdd5c42fa03bb97030d71e3f"} Jul 27 21:29:25.659660 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:25.659623 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:25.660937 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:25.660901 2565 patch_prober.go:28] interesting pod/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 27 21:29:25.661200 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:25.661167 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" podUID="7856196d-a32c-400c-b11d-eff4d8c24fb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 27 21:29:26.038042 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:26.038006 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:26.660084 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:26.660048 2565 patch_prober.go:28] interesting pod/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 27 21:29:26.660590 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:26.660101 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" podUID="7856196d-a32c-400c-b11d-eff4d8c24fb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 27 21:29:27.660059 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:27.660015 2565 patch_prober.go:28] interesting pod/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 27 21:29:27.660268 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:27.660080 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" podUID="7856196d-a32c-400c-b11d-eff4d8c24fb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 27 21:29:28.659912 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:28.659879 2565 patch_prober.go:28] interesting pod/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 27 21:29:28.660081 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:28.659932 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" podUID="7856196d-a32c-400c-b11d-eff4d8c24fb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 27 21:29:29.673768 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:29.673729 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:29.680354 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:29.680318 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:29:36.145404 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:36.145365 2565 patch_prober.go:28] interesting pod/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 27 21:29:36.145799 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:36.145430 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" podUID="f672c708-8e8c-4489-b3a7-25c4e85be824" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 27 21:29:37.215555 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:37.215518 2565 generic.go:356] "Generic (PLEG): container finished" podID="f672c708-8e8c-4489-b3a7-25c4e85be824" containerID="df7b892aea3343579095db64f93861420a1ef6a6e746968935b2216cf5ab5f14" exitCode=0 Jul 27 21:29:37.216003 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:37.215568 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" event={"ID":"f672c708-8e8c-4489-b3a7-25c4e85be824","Type":"ContainerDied","Data":"df7b892aea3343579095db64f93861420a1ef6a6e746968935b2216cf5ab5f14"} Jul 27 21:29:37.216003 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:37.215926 2565 scope.go:117] "RemoveContainer" containerID="df7b892aea3343579095db64f93861420a1ef6a6e746968935b2216cf5ab5f14" Jul 27 21:29:37.217002 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:37.216974 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:29:37.217112 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:37.217045 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:29:37.217112 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:37.217082 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:29:38.222606 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:38.222571 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" event={"ID":"f672c708-8e8c-4489-b3a7-25c4e85be824","Type":"ContainerStarted","Data":"5c4b4434364cfa023b3d52efb97ca86be90be224de400db39350e7594f76a741"} Jul 27 21:29:38.818076 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:38.818032 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:38.818326 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:38.818303 2565 patch_prober.go:28] interesting pod/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 27 21:29:38.818380 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:38.818351 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" podUID="f672c708-8e8c-4489-b3a7-25c4e85be824" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 27 21:29:39.818187 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:39.818147 2565 patch_prober.go:28] interesting pod/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 27 21:29:39.818664 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:39.818213 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" podUID="f672c708-8e8c-4489-b3a7-25c4e85be824" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 27 21:29:40.818444 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:40.818408 2565 patch_prober.go:28] interesting pod/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 27 21:29:40.818825 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:40.818468 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" podUID="f672c708-8e8c-4489-b3a7-25c4e85be824" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 27 21:29:41.818363 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:41.818323 2565 patch_prober.go:28] interesting pod/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 27 21:29:41.818542 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:41.818404 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" podUID="f672c708-8e8c-4489-b3a7-25c4e85be824" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 27 21:29:42.822010 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:42.821976 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:42.822503 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:42.822481 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:42.823099 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:42.823077 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:29:59.973317 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:59.973281 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9"] Jul 27 21:29:59.973827 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:29:59.973546 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" podUID="7856196d-a32c-400c-b11d-eff4d8c24fb6" containerName="istio-proxy" containerID="cri-o://3d082289801b61dfb60c521d437a04ced876a75bcdd5c42fa03bb97030d71e3f" gracePeriod=30 Jul 27 21:30:05.228512 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.228479 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:30:05.325668 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.325570 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-data\") pod \"7856196d-a32c-400c-b11d-eff4d8c24fb6\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " Jul 27 21:30:05.325668 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.325618 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-podinfo\") pod \"7856196d-a32c-400c-b11d-eff4d8c24fb6\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " Jul 27 21:30:05.325668 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.325643 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-envoy\") pod \"7856196d-a32c-400c-b11d-eff4d8c24fb6\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " Jul 27 21:30:05.325668 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.325670 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-workload-certs\") pod \"7856196d-a32c-400c-b11d-eff4d8c24fb6\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " Jul 27 21:30:05.326003 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.325692 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7856196d-a32c-400c-b11d-eff4d8c24fb6-istiod-ca-cert\") pod \"7856196d-a32c-400c-b11d-eff4d8c24fb6\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " Jul 27 21:30:05.326003 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.325713 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-workload-socket\") pod \"7856196d-a32c-400c-b11d-eff4d8c24fb6\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " Jul 27 21:30:05.326003 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.325789 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-credential-socket\") pod \"7856196d-a32c-400c-b11d-eff4d8c24fb6\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " Jul 27 21:30:05.326003 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.325817 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-token\") pod \"7856196d-a32c-400c-b11d-eff4d8c24fb6\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " Jul 27 21:30:05.326003 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.325887 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7gtf\" (UniqueName: \"kubernetes.io/projected/7856196d-a32c-400c-b11d-eff4d8c24fb6-kube-api-access-h7gtf\") pod \"7856196d-a32c-400c-b11d-eff4d8c24fb6\" (UID: \"7856196d-a32c-400c-b11d-eff4d8c24fb6\") " Jul 27 21:30:05.326297 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.326021 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7856196d-a32c-400c-b11d-eff4d8c24fb6" (UID: "7856196d-a32c-400c-b11d-eff4d8c24fb6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:30:05.326297 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.326211 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7856196d-a32c-400c-b11d-eff4d8c24fb6" (UID: "7856196d-a32c-400c-b11d-eff4d8c24fb6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:30:05.326297 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.326224 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7856196d-a32c-400c-b11d-eff4d8c24fb6" (UID: "7856196d-a32c-400c-b11d-eff4d8c24fb6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:30:05.326297 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.326264 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-workload-certs\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:30:05.326548 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.326466 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-data" (OuterVolumeSpecName: "istio-data") pod "7856196d-a32c-400c-b11d-eff4d8c24fb6" (UID: "7856196d-a32c-400c-b11d-eff4d8c24fb6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:30:05.326617 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.326597 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7856196d-a32c-400c-b11d-eff4d8c24fb6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7856196d-a32c-400c-b11d-eff4d8c24fb6" (UID: "7856196d-a32c-400c-b11d-eff4d8c24fb6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 21:30:05.328049 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.328026 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7856196d-a32c-400c-b11d-eff4d8c24fb6" (UID: "7856196d-a32c-400c-b11d-eff4d8c24fb6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 27 21:30:05.328049 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.328024 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7856196d-a32c-400c-b11d-eff4d8c24fb6" (UID: "7856196d-a32c-400c-b11d-eff4d8c24fb6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:30:05.328677 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.328655 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7856196d-a32c-400c-b11d-eff4d8c24fb6-kube-api-access-h7gtf" (OuterVolumeSpecName: "kube-api-access-h7gtf") pod "7856196d-a32c-400c-b11d-eff4d8c24fb6" (UID: "7856196d-a32c-400c-b11d-eff4d8c24fb6"). InnerVolumeSpecName "kube-api-access-h7gtf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:30:05.328776 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.328680 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-token" (OuterVolumeSpecName: "istio-token") pod "7856196d-a32c-400c-b11d-eff4d8c24fb6" (UID: "7856196d-a32c-400c-b11d-eff4d8c24fb6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:30:05.355500 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.355467 2565 generic.go:356] "Generic (PLEG): container finished" podID="7856196d-a32c-400c-b11d-eff4d8c24fb6" containerID="3d082289801b61dfb60c521d437a04ced876a75bcdd5c42fa03bb97030d71e3f" exitCode=0 Jul 27 21:30:05.355622 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.355515 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" Jul 27 21:30:05.355622 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.355535 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" event={"ID":"7856196d-a32c-400c-b11d-eff4d8c24fb6","Type":"ContainerDied","Data":"3d082289801b61dfb60c521d437a04ced876a75bcdd5c42fa03bb97030d71e3f"} Jul 27 21:30:05.355622 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.355590 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9" event={"ID":"7856196d-a32c-400c-b11d-eff4d8c24fb6","Type":"ContainerDied","Data":"826f478f7044db95123a97e594a889cb79c2efa53b8b16b8ec12ea4d00940840"} Jul 27 21:30:05.355622 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.355612 2565 scope.go:117] "RemoveContainer" containerID="3d082289801b61dfb60c521d437a04ced876a75bcdd5c42fa03bb97030d71e3f" Jul 27 21:30:05.365052 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.365035 2565 scope.go:117] "RemoveContainer" containerID="e74a41f05645741ff1e50bddc623e54e44e904fdab710021feba6bd7d09ae73c" Jul 27 21:30:05.373829 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.373812 2565 scope.go:117] "RemoveContainer" containerID="3d082289801b61dfb60c521d437a04ced876a75bcdd5c42fa03bb97030d71e3f" Jul 27 21:30:05.374101 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:30:05.374082 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d082289801b61dfb60c521d437a04ced876a75bcdd5c42fa03bb97030d71e3f\": container with ID starting with 3d082289801b61dfb60c521d437a04ced876a75bcdd5c42fa03bb97030d71e3f not found: ID does not exist" containerID="3d082289801b61dfb60c521d437a04ced876a75bcdd5c42fa03bb97030d71e3f" Jul 27 21:30:05.374186 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.374113 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d082289801b61dfb60c521d437a04ced876a75bcdd5c42fa03bb97030d71e3f"} err="failed to get container status \"3d082289801b61dfb60c521d437a04ced876a75bcdd5c42fa03bb97030d71e3f\": rpc error: code = NotFound desc = could not find container \"3d082289801b61dfb60c521d437a04ced876a75bcdd5c42fa03bb97030d71e3f\": container with ID starting with 3d082289801b61dfb60c521d437a04ced876a75bcdd5c42fa03bb97030d71e3f not found: ID does not exist" Jul 27 21:30:05.374186 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.374136 2565 scope.go:117] "RemoveContainer" containerID="e74a41f05645741ff1e50bddc623e54e44e904fdab710021feba6bd7d09ae73c" Jul 27 21:30:05.374418 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:30:05.374400 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e74a41f05645741ff1e50bddc623e54e44e904fdab710021feba6bd7d09ae73c\": container with ID starting with e74a41f05645741ff1e50bddc623e54e44e904fdab710021feba6bd7d09ae73c not found: ID does not exist" containerID="e74a41f05645741ff1e50bddc623e54e44e904fdab710021feba6bd7d09ae73c" Jul 27 21:30:05.374458 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.374423 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e74a41f05645741ff1e50bddc623e54e44e904fdab710021feba6bd7d09ae73c"} err="failed to get container status \"e74a41f05645741ff1e50bddc623e54e44e904fdab710021feba6bd7d09ae73c\": rpc error: code = NotFound desc = could not find container \"e74a41f05645741ff1e50bddc623e54e44e904fdab710021feba6bd7d09ae73c\": container with ID starting with e74a41f05645741ff1e50bddc623e54e44e904fdab710021feba6bd7d09ae73c not found: ID does not exist" Jul 27 21:30:05.378034 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.378013 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9"] Jul 27 21:30:05.384494 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.384473 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-c3ee91-openshift-default-987b54fc6-jvsk9"] Jul 27 21:30:05.427403 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.427374 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-credential-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:30:05.427403 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.427398 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-token\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:30:05.427403 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.427409 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7gtf\" (UniqueName: \"kubernetes.io/projected/7856196d-a32c-400c-b11d-eff4d8c24fb6-kube-api-access-h7gtf\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:30:05.427695 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.427417 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-data\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:30:05.427695 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.427428 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-podinfo\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:30:05.427695 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.427438 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-istio-envoy\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:30:05.427695 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.427445 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7856196d-a32c-400c-b11d-eff4d8c24fb6-istiod-ca-cert\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:30:05.427695 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:05.427453 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7856196d-a32c-400c-b11d-eff4d8c24fb6-workload-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:30:06.051887 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:06.051847 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7856196d-a32c-400c-b11d-eff4d8c24fb6" path="/var/lib/kubelet/pods/7856196d-a32c-400c-b11d-eff4d8c24fb6/volumes" Jul 27 21:30:27.116793 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:27.116755 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt"] Jul 27 21:30:27.117223 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:27.117002 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" podUID="f672c708-8e8c-4489-b3a7-25c4e85be824" containerName="istio-proxy" containerID="cri-o://5c4b4434364cfa023b3d52efb97ca86be90be224de400db39350e7594f76a741" gracePeriod=30 Jul 27 21:30:27.825328 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:27.825288 2565 patch_prober.go:28] interesting pod/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 27 21:30:27.825496 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:27.825350 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" podUID="f672c708-8e8c-4489-b3a7-25c4e85be824" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 27 21:30:27.986442 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:27.986404 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll"] Jul 27 21:30:27.986764 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:27.986751 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7856196d-a32c-400c-b11d-eff4d8c24fb6" containerName="istio-proxy" Jul 27 21:30:27.986808 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:27.986765 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7856196d-a32c-400c-b11d-eff4d8c24fb6" containerName="istio-proxy" Jul 27 21:30:27.986808 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:27.986776 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7856196d-a32c-400c-b11d-eff4d8c24fb6" containerName="istio-proxy" Jul 27 21:30:27.986808 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:27.986781 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7856196d-a32c-400c-b11d-eff4d8c24fb6" containerName="istio-proxy" Jul 27 21:30:27.986905 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:27.986839 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7856196d-a32c-400c-b11d-eff4d8c24fb6" containerName="istio-proxy" Jul 27 21:30:27.986905 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:27.986848 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7856196d-a32c-400c-b11d-eff4d8c24fb6" containerName="istio-proxy" Jul 27 21:30:27.989682 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:27.989665 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:27.992389 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:27.992367 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-c910ce-openshift-default-dockercfg-7zchp\"" Jul 27 21:30:28.005339 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.005304 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll"] Jul 27 21:30:28.133610 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.133501 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-workload-certs\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.133610 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.133558 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-credential-socket\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.133610 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.133578 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-envoy\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.134070 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.133612 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istiod-ca-cert\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.134070 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.133636 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmjlk\" (UniqueName: \"kubernetes.io/projected/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-kube-api-access-hmjlk\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.134070 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.133698 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-podinfo\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.134070 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.133729 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-token\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.134070 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.133810 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-workload-socket\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.134070 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.133857 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-data\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.235041 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.234996 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-podinfo\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.235277 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.235051 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-token\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.235277 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.235078 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-workload-socket\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.235277 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.235106 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-data\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.235277 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.235136 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-workload-certs\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.235277 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.235158 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-credential-socket\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.235277 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.235175 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-envoy\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.235591 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.235497 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istiod-ca-cert\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.235591 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.235564 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmjlk\" (UniqueName: \"kubernetes.io/projected/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-kube-api-access-hmjlk\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.235682 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.235649 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-data\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.235766 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.235746 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-workload-socket\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.235853 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.235777 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-workload-certs\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.235853 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.235828 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-credential-socket\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.236177 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.236150 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istiod-ca-cert\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.237808 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.237784 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-envoy\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.238034 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.238009 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-podinfo\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.243879 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.243856 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-token\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.244164 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.244142 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmjlk\" (UniqueName: \"kubernetes.io/projected/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-kube-api-access-hmjlk\") pod \"e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.300679 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.300618 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:28.442382 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.442356 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll"] Jul 27 21:30:28.444722 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:30:28.444693 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfaa5a15_c1e7_4ab4_bf7e_5ead0e6c819d.slice/crio-abac92e1b03d1effff196c48d08431bcecf124bec0521c65c9fe28ff1e110b92 WatchSource:0}: Error finding container abac92e1b03d1effff196c48d08431bcecf124bec0521c65c9fe28ff1e110b92: Status 404 returned error can't find the container with id abac92e1b03d1effff196c48d08431bcecf124bec0521c65c9fe28ff1e110b92 Jul 27 21:30:28.447047 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.447008 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:30:28.447147 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.447094 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:30:28.447147 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.447141 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:30:28.473278 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:28.473214 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" event={"ID":"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d","Type":"ContainerStarted","Data":"abac92e1b03d1effff196c48d08431bcecf124bec0521c65c9fe28ff1e110b92"} Jul 27 21:30:29.479030 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:29.478991 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" event={"ID":"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d","Type":"ContainerStarted","Data":"5cdf00c6ea91906160b165686b642c2490832eb9dd25f8f087ef522bbdf29f02"} Jul 27 21:30:29.512781 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:29.512724 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" podStartSLOduration=2.5127070270000003 podStartE2EDuration="2.512707027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:30:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:30:29.510568632 +0000 UTC m=+2358.033686354" watchObservedRunningTime="2026-07-27 21:30:29.512707027 +0000 UTC m=+2358.035824748" Jul 27 21:30:30.301461 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:30.301430 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:30.307065 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:30.307038 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:30.484317 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:30.484287 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:30.485457 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:30.485434 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:32.362273 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.362229 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:30:32.477388 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.477352 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-data\") pod \"f672c708-8e8c-4489-b3a7-25c4e85be824\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " Jul 27 21:30:32.477388 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.477397 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f672c708-8e8c-4489-b3a7-25c4e85be824-istiod-ca-cert\") pod \"f672c708-8e8c-4489-b3a7-25c4e85be824\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " Jul 27 21:30:32.477644 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.477438 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-credential-socket\") pod \"f672c708-8e8c-4489-b3a7-25c4e85be824\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " Jul 27 21:30:32.477644 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.477509 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-workload-certs\") pod \"f672c708-8e8c-4489-b3a7-25c4e85be824\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " Jul 27 21:30:32.477644 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.477537 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-envoy\") pod \"f672c708-8e8c-4489-b3a7-25c4e85be824\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " Jul 27 21:30:32.477644 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.477575 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-podinfo\") pod \"f672c708-8e8c-4489-b3a7-25c4e85be824\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " Jul 27 21:30:32.477644 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.477609 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-workload-socket\") pod \"f672c708-8e8c-4489-b3a7-25c4e85be824\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " Jul 27 21:30:32.477889 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.477652 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-token\") pod \"f672c708-8e8c-4489-b3a7-25c4e85be824\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " Jul 27 21:30:32.477889 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.477683 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8hm9\" (UniqueName: \"kubernetes.io/projected/f672c708-8e8c-4489-b3a7-25c4e85be824-kube-api-access-s8hm9\") pod \"f672c708-8e8c-4489-b3a7-25c4e85be824\" (UID: \"f672c708-8e8c-4489-b3a7-25c4e85be824\") " Jul 27 21:30:32.477889 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.477855 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f672c708-8e8c-4489-b3a7-25c4e85be824-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f672c708-8e8c-4489-b3a7-25c4e85be824" (UID: "f672c708-8e8c-4489-b3a7-25c4e85be824"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 21:30:32.478198 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.478106 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f672c708-8e8c-4489-b3a7-25c4e85be824-istiod-ca-cert\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:30:32.478449 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.478408 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f672c708-8e8c-4489-b3a7-25c4e85be824" (UID: "f672c708-8e8c-4489-b3a7-25c4e85be824"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:30:32.478720 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.478534 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-data" (OuterVolumeSpecName: "istio-data") pod "f672c708-8e8c-4489-b3a7-25c4e85be824" (UID: "f672c708-8e8c-4489-b3a7-25c4e85be824"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:30:32.478824 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.478730 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f672c708-8e8c-4489-b3a7-25c4e85be824" (UID: "f672c708-8e8c-4489-b3a7-25c4e85be824"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:30:32.478824 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.478780 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f672c708-8e8c-4489-b3a7-25c4e85be824" (UID: "f672c708-8e8c-4489-b3a7-25c4e85be824"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:30:32.482548 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.482460 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f672c708-8e8c-4489-b3a7-25c4e85be824-kube-api-access-s8hm9" (OuterVolumeSpecName: "kube-api-access-s8hm9") pod "f672c708-8e8c-4489-b3a7-25c4e85be824" (UID: "f672c708-8e8c-4489-b3a7-25c4e85be824"). InnerVolumeSpecName "kube-api-access-s8hm9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:30:32.482548 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.482523 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f672c708-8e8c-4489-b3a7-25c4e85be824" (UID: "f672c708-8e8c-4489-b3a7-25c4e85be824"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:30:32.483180 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.483145 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-token" (OuterVolumeSpecName: "istio-token") pod "f672c708-8e8c-4489-b3a7-25c4e85be824" (UID: "f672c708-8e8c-4489-b3a7-25c4e85be824"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:30:32.484379 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.484353 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f672c708-8e8c-4489-b3a7-25c4e85be824" (UID: "f672c708-8e8c-4489-b3a7-25c4e85be824"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 27 21:30:32.496936 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.496907 2565 generic.go:356] "Generic (PLEG): container finished" podID="f672c708-8e8c-4489-b3a7-25c4e85be824" containerID="5c4b4434364cfa023b3d52efb97ca86be90be224de400db39350e7594f76a741" exitCode=0 Jul 27 21:30:32.497066 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.496971 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" Jul 27 21:30:32.497066 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.496999 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" event={"ID":"f672c708-8e8c-4489-b3a7-25c4e85be824","Type":"ContainerDied","Data":"5c4b4434364cfa023b3d52efb97ca86be90be224de400db39350e7594f76a741"} Jul 27 21:30:32.497141 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.497061 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt" event={"ID":"f672c708-8e8c-4489-b3a7-25c4e85be824","Type":"ContainerDied","Data":"c2e8f19e2e9481f871bb47c57e106e0f97efe28299f20b9dcaa18b6b2929c9fa"} Jul 27 21:30:32.497141 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.497098 2565 scope.go:117] "RemoveContainer" containerID="5c4b4434364cfa023b3d52efb97ca86be90be224de400db39350e7594f76a741" Jul 27 21:30:32.513881 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.513858 2565 scope.go:117] "RemoveContainer" containerID="df7b892aea3343579095db64f93861420a1ef6a6e746968935b2216cf5ab5f14" Jul 27 21:30:32.522692 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.522656 2565 scope.go:117] "RemoveContainer" containerID="5c4b4434364cfa023b3d52efb97ca86be90be224de400db39350e7594f76a741" Jul 27 21:30:32.523112 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:30:32.523077 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c4b4434364cfa023b3d52efb97ca86be90be224de400db39350e7594f76a741\": container with ID starting with 5c4b4434364cfa023b3d52efb97ca86be90be224de400db39350e7594f76a741 not found: ID does not exist" containerID="5c4b4434364cfa023b3d52efb97ca86be90be224de400db39350e7594f76a741" Jul 27 21:30:32.523174 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.523139 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c4b4434364cfa023b3d52efb97ca86be90be224de400db39350e7594f76a741"} err="failed to get container status \"5c4b4434364cfa023b3d52efb97ca86be90be224de400db39350e7594f76a741\": rpc error: code = NotFound desc = could not find container \"5c4b4434364cfa023b3d52efb97ca86be90be224de400db39350e7594f76a741\": container with ID starting with 5c4b4434364cfa023b3d52efb97ca86be90be224de400db39350e7594f76a741 not found: ID does not exist" Jul 27 21:30:32.523265 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.523182 2565 scope.go:117] "RemoveContainer" containerID="df7b892aea3343579095db64f93861420a1ef6a6e746968935b2216cf5ab5f14" Jul 27 21:30:32.523585 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:30:32.523562 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df7b892aea3343579095db64f93861420a1ef6a6e746968935b2216cf5ab5f14\": container with ID starting with df7b892aea3343579095db64f93861420a1ef6a6e746968935b2216cf5ab5f14 not found: ID does not exist" containerID="df7b892aea3343579095db64f93861420a1ef6a6e746968935b2216cf5ab5f14" Jul 27 21:30:32.523647 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.523590 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df7b892aea3343579095db64f93861420a1ef6a6e746968935b2216cf5ab5f14"} err="failed to get container status \"df7b892aea3343579095db64f93861420a1ef6a6e746968935b2216cf5ab5f14\": rpc error: code = NotFound desc = could not find container \"df7b892aea3343579095db64f93861420a1ef6a6e746968935b2216cf5ab5f14\": container with ID starting with df7b892aea3343579095db64f93861420a1ef6a6e746968935b2216cf5ab5f14 not found: ID does not exist" Jul 27 21:30:32.528483 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.528461 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt"] Jul 27 21:30:32.531462 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.531432 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-86cc77-openshift-default-79cf45f494-297tt"] Jul 27 21:30:32.579283 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.579216 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-envoy\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:30:32.579283 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.579274 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-podinfo\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:30:32.579525 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.579302 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-workload-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:30:32.579525 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.579318 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-token\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:30:32.579525 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.579332 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8hm9\" (UniqueName: \"kubernetes.io/projected/f672c708-8e8c-4489-b3a7-25c4e85be824-kube-api-access-s8hm9\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:30:32.579525 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.579345 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-istio-data\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:30:32.579525 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.579357 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-credential-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:30:32.579525 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:32.579368 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f672c708-8e8c-4489-b3a7-25c4e85be824-workload-certs\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:30:34.053356 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:34.053327 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f672c708-8e8c-4489-b3a7-25c4e85be824" path="/var/lib/kubelet/pods/f672c708-8e8c-4489-b3a7-25c4e85be824/volumes" Jul 27 21:30:36.545030 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.545000 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm"] Jul 27 21:30:36.545506 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.545370 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f672c708-8e8c-4489-b3a7-25c4e85be824" containerName="istio-proxy" Jul 27 21:30:36.545506 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.545382 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f672c708-8e8c-4489-b3a7-25c4e85be824" containerName="istio-proxy" Jul 27 21:30:36.545506 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.545395 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f672c708-8e8c-4489-b3a7-25c4e85be824" containerName="istio-proxy" Jul 27 21:30:36.545506 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.545401 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f672c708-8e8c-4489-b3a7-25c4e85be824" containerName="istio-proxy" Jul 27 21:30:36.545506 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.545455 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f672c708-8e8c-4489-b3a7-25c4e85be824" containerName="istio-proxy" Jul 27 21:30:36.545506 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.545464 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f672c708-8e8c-4489-b3a7-25c4e85be824" containerName="istio-proxy" Jul 27 21:30:36.548635 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.548617 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.551991 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.551969 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-85f004-openshift-default-dockercfg-ccscf\"" Jul 27 21:30:36.561689 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.561663 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm"] Jul 27 21:30:36.613717 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.613680 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-podinfo\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.613717 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.613722 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-data\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.613960 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.613755 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-workload-socket\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.613960 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.613839 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-envoy\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.613960 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.613880 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq8wh\" (UniqueName: \"kubernetes.io/projected/afc1af5c-6ffa-491c-9ffa-cebb5d217775-kube-api-access-tq8wh\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.613960 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.613907 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-credential-socket\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.613960 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.613923 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istiod-ca-cert\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.613960 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.613960 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-token\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.614160 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.614000 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-workload-certs\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.715295 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.715255 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-workload-certs\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.715295 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.715291 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-podinfo\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.715556 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.715319 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-data\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.715556 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.715452 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-workload-socket\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.715670 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.715551 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-envoy\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.715670 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.715584 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq8wh\" (UniqueName: \"kubernetes.io/projected/afc1af5c-6ffa-491c-9ffa-cebb5d217775-kube-api-access-tq8wh\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.715670 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.715627 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-credential-socket\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.715670 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.715656 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istiod-ca-cert\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.715875 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.715676 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-data\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.715875 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.715698 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-workload-certs\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.715875 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.715714 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-token\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.715875 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.715766 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-workload-socket\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.716097 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.715898 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-credential-socket\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.716337 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.716321 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istiod-ca-cert\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.717928 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.717905 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-envoy\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.718035 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.717961 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-podinfo\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.726861 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.726828 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-token\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.726861 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.726844 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq8wh\" (UniqueName: \"kubernetes.io/projected/afc1af5c-6ffa-491c-9ffa-cebb5d217775-kube-api-access-tq8wh\") pod \"e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:36.858634 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:36.858548 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:37.011555 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:37.011522 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm"] Jul 27 21:30:37.013181 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:30:37.013156 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafc1af5c_6ffa_491c_9ffa_cebb5d217775.slice/crio-c222cccb38de11acaf254715eda283d19c12ee15e80e9d78f3f1ed19dfcbfcdc WatchSource:0}: Error finding container c222cccb38de11acaf254715eda283d19c12ee15e80e9d78f3f1ed19dfcbfcdc: Status 404 returned error can't find the container with id c222cccb38de11acaf254715eda283d19c12ee15e80e9d78f3f1ed19dfcbfcdc Jul 27 21:30:37.015341 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:37.015304 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:30:37.015457 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:37.015395 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:30:37.015457 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:37.015440 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:30:37.525518 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:37.525479 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" event={"ID":"afc1af5c-6ffa-491c-9ffa-cebb5d217775","Type":"ContainerStarted","Data":"0bab1f508bd05cd8df1fd576170270bdc37c2481bf082efb98535bc0b0302067"} Jul 27 21:30:37.525727 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:37.525532 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" event={"ID":"afc1af5c-6ffa-491c-9ffa-cebb5d217775","Type":"ContainerStarted","Data":"c222cccb38de11acaf254715eda283d19c12ee15e80e9d78f3f1ed19dfcbfcdc"} Jul 27 21:30:37.550793 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:37.550732 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" podStartSLOduration=1.5507177909999998 podStartE2EDuration="1.550717791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:30:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:30:37.548135758 +0000 UTC m=+2366.071253480" watchObservedRunningTime="2026-07-27 21:30:37.550717791 +0000 UTC m=+2366.073835513" Jul 27 21:30:37.858894 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:37.858804 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:38.864468 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:38.864435 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:39.537785 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:39.537754 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:39.538823 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:39.538802 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:46.526454 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:46.526408 2565 patch_prober.go:28] interesting pod/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 27 21:30:46.526918 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:46.526500 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" podUID="bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 27 21:30:47.587040 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:47.586919 2565 generic.go:356] "Generic (PLEG): container finished" podID="bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" containerID="5cdf00c6ea91906160b165686b642c2490832eb9dd25f8f087ef522bbdf29f02" exitCode=0 Jul 27 21:30:47.587594 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:47.587076 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" event={"ID":"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d","Type":"ContainerDied","Data":"5cdf00c6ea91906160b165686b642c2490832eb9dd25f8f087ef522bbdf29f02"} Jul 27 21:30:47.587667 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:47.587603 2565 scope.go:117] "RemoveContainer" containerID="5cdf00c6ea91906160b165686b642c2490832eb9dd25f8f087ef522bbdf29f02" Jul 27 21:30:47.588782 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:47.588754 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:30:47.588838 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:47.588824 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:30:47.588874 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:47.588857 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:30:48.598541 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:48.597570 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" event={"ID":"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d","Type":"ContainerStarted","Data":"4bcd0443a87004ea4d2118a1056cdeaeaf325149f2316589ae77ddee3434cd0d"} Jul 27 21:30:49.302062 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:49.302017 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:49.302452 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:49.302097 2565 patch_prober.go:28] interesting pod/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 27 21:30:49.302568 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:49.302490 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" podUID="bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 27 21:30:50.302287 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:50.301759 2565 patch_prober.go:28] interesting pod/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 27 21:30:50.302287 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:50.301841 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" podUID="bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 27 21:30:51.302261 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:51.301726 2565 patch_prober.go:28] interesting pod/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 27 21:30:51.302261 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:51.301811 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" podUID="bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 27 21:30:52.301599 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:52.301553 2565 patch_prober.go:28] interesting pod/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 27 21:30:52.301812 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:52.301614 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" podUID="bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 27 21:30:53.304496 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:53.304447 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:53.304989 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:53.304968 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:53.305671 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:53.305648 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:30:54.588860 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:54.588650 2565 patch_prober.go:28] interesting pod/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:44140->10.132.0.56:15021: read: connection reset by peer" start-of-body= Jul 27 21:30:54.588860 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:54.588745 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" podUID="afc1af5c-6ffa-491c-9ffa-cebb5d217775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:44140->10.132.0.56:15021: read: connection reset by peer" Jul 27 21:30:54.634000 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:54.633966 2565 generic.go:356] "Generic (PLEG): container finished" podID="afc1af5c-6ffa-491c-9ffa-cebb5d217775" containerID="0bab1f508bd05cd8df1fd576170270bdc37c2481bf082efb98535bc0b0302067" exitCode=0 Jul 27 21:30:54.634126 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:54.634041 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" event={"ID":"afc1af5c-6ffa-491c-9ffa-cebb5d217775","Type":"ContainerDied","Data":"0bab1f508bd05cd8df1fd576170270bdc37c2481bf082efb98535bc0b0302067"} Jul 27 21:30:54.634503 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:54.634484 2565 scope.go:117] "RemoveContainer" containerID="0bab1f508bd05cd8df1fd576170270bdc37c2481bf082efb98535bc0b0302067" Jul 27 21:30:54.635627 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:54.635602 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:30:54.635687 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:54.635672 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:30:54.635724 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:54.635704 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:30:55.642800 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:55.642767 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" event={"ID":"afc1af5c-6ffa-491c-9ffa-cebb5d217775","Type":"ContainerStarted","Data":"e7bec236d316e032a5b6d866c7ec799fa2e1a63ad319d6f6dbace3d29dae676e"} Jul 27 21:30:55.858719 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:55.858685 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:55.858929 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:55.858910 2565 patch_prober.go:28] interesting pod/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 27 21:30:55.858991 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:55.858952 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" podUID="afc1af5c-6ffa-491c-9ffa-cebb5d217775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 27 21:30:56.858935 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:56.858901 2565 patch_prober.go:28] interesting pod/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 27 21:30:56.859336 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:56.858953 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" podUID="afc1af5c-6ffa-491c-9ffa-cebb5d217775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 27 21:30:57.858984 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:57.858950 2565 patch_prober.go:28] interesting pod/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 27 21:30:57.859387 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:57.859008 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" podUID="afc1af5c-6ffa-491c-9ffa-cebb5d217775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 27 21:30:58.859606 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:58.859572 2565 patch_prober.go:28] interesting pod/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 27 21:30:58.859990 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:58.859633 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" podUID="afc1af5c-6ffa-491c-9ffa-cebb5d217775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 27 21:30:59.863381 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:59.863345 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:59.863859 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:59.863844 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:30:59.864511 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:30:59.864489 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:31:28.056090 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.056057 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9"] Jul 27 21:31:28.059352 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.059335 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.062205 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.062183 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-85f004\"/\"openshift-service-ca.crt\"" Jul 27 21:31:28.063525 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.063505 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-85f004\"/\"default-dockercfg-59fsx\"" Jul 27 21:31:28.063641 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.063554 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-85f004\"/\"auth-test-model-85f004-kserve-self-signed-certs\"" Jul 27 21:31:28.063641 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.063586 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-85f004\"/\"kube-root-ca.crt\"" Jul 27 21:31:28.074428 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.074405 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9"] Jul 27 21:31:28.110970 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.110931 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2qlk\" (UniqueName: \"kubernetes.io/projected/cef9790b-68a2-4d22-8395-cbc48af58942-kube-api-access-m2qlk\") pod \"auth-test-model-85f004-kserve-6b5d59f75-2bqw9\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.111139 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.110987 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-model-cache\") pod \"auth-test-model-85f004-kserve-6b5d59f75-2bqw9\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.111139 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.111030 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-tmp-dir\") pod \"auth-test-model-85f004-kserve-6b5d59f75-2bqw9\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.111139 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.111117 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-dshm\") pod \"auth-test-model-85f004-kserve-6b5d59f75-2bqw9\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.111139 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.111136 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cef9790b-68a2-4d22-8395-cbc48af58942-tls-certs\") pod \"auth-test-model-85f004-kserve-6b5d59f75-2bqw9\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.111377 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.111204 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-home\") pod \"auth-test-model-85f004-kserve-6b5d59f75-2bqw9\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.212612 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.212563 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-dshm\") pod \"auth-test-model-85f004-kserve-6b5d59f75-2bqw9\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.212612 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.212614 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cef9790b-68a2-4d22-8395-cbc48af58942-tls-certs\") pod \"auth-test-model-85f004-kserve-6b5d59f75-2bqw9\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.212894 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.212639 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-home\") pod \"auth-test-model-85f004-kserve-6b5d59f75-2bqw9\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.212894 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.212659 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2qlk\" (UniqueName: \"kubernetes.io/projected/cef9790b-68a2-4d22-8395-cbc48af58942-kube-api-access-m2qlk\") pod \"auth-test-model-85f004-kserve-6b5d59f75-2bqw9\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.212894 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.212769 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-model-cache\") pod \"auth-test-model-85f004-kserve-6b5d59f75-2bqw9\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.212894 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.212820 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-tmp-dir\") pod \"auth-test-model-85f004-kserve-6b5d59f75-2bqw9\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.213137 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.213121 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-model-cache\") pod \"auth-test-model-85f004-kserve-6b5d59f75-2bqw9\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.213204 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.213185 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-home\") pod \"auth-test-model-85f004-kserve-6b5d59f75-2bqw9\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.213278 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.213225 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-tmp-dir\") pod \"auth-test-model-85f004-kserve-6b5d59f75-2bqw9\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.215067 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.215048 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-dshm\") pod \"auth-test-model-85f004-kserve-6b5d59f75-2bqw9\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.215358 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.215341 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cef9790b-68a2-4d22-8395-cbc48af58942-tls-certs\") pod \"auth-test-model-85f004-kserve-6b5d59f75-2bqw9\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.222626 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.222601 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2qlk\" (UniqueName: \"kubernetes.io/projected/cef9790b-68a2-4d22-8395-cbc48af58942-kube-api-access-m2qlk\") pod \"auth-test-model-85f004-kserve-6b5d59f75-2bqw9\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.368900 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.368809 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:28.509865 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.509834 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9"] Jul 27 21:31:28.512038 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:31:28.512004 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcef9790b_68a2_4d22_8395_cbc48af58942.slice/crio-0c2ee2e1fabf79efabd6cd8d58c0ba7dbf5655ac8106bd4c24bbbe78d7ebb7da WatchSource:0}: Error finding container 0c2ee2e1fabf79efabd6cd8d58c0ba7dbf5655ac8106bd4c24bbbe78d7ebb7da: Status 404 returned error can't find the container with id 0c2ee2e1fabf79efabd6cd8d58c0ba7dbf5655ac8106bd4c24bbbe78d7ebb7da Jul 27 21:31:28.809825 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.809785 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" event={"ID":"cef9790b-68a2-4d22-8395-cbc48af58942","Type":"ContainerStarted","Data":"10e47f4574442a6b4fac74ee894de51d583750e4fe5369f531d1b78f3b58a6e7"} Jul 27 21:31:28.810011 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.809836 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" event={"ID":"cef9790b-68a2-4d22-8395-cbc48af58942","Type":"ContainerStarted","Data":"0c2ee2e1fabf79efabd6cd8d58c0ba7dbf5655ac8106bd4c24bbbe78d7ebb7da"} Jul 27 21:31:28.831538 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:28.831487 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" podStartSLOduration=0.831471395 podStartE2EDuration="831.471395ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:31:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:31:28.829579213 +0000 UTC m=+2417.352696925" watchObservedRunningTime="2026-07-27 21:31:28.831471395 +0000 UTC m=+2417.354589116" Jul 27 21:31:38.369298 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:38.369204 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:38.369298 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:38.369283 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:38.381622 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:38.381586 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:31:38.868594 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:31:38.868566 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:35:40.423248 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:40.423210 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9"] Jul 27 21:35:55.442562 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.442521 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" podUID="cef9790b-68a2-4d22-8395-cbc48af58942" containerName="main" containerID="cri-o://10e47f4574442a6b4fac74ee894de51d583750e4fe5369f531d1b78f3b58a6e7" gracePeriod=45 Jul 27 21:35:55.472334 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:35:55.472306 2565 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcef9790b_68a2_4d22_8395_cbc48af58942.slice/crio-10e47f4574442a6b4fac74ee894de51d583750e4fe5369f531d1b78f3b58a6e7.scope\": RecentStats: unable to find data in memory cache]" Jul 27 21:35:55.697745 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.697684 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:35:55.766950 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.766920 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-model-cache\") pod \"cef9790b-68a2-4d22-8395-cbc48af58942\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " Jul 27 21:35:55.766950 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.766953 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-tmp-dir\") pod \"cef9790b-68a2-4d22-8395-cbc48af58942\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " Jul 27 21:35:55.767183 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.766979 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-dshm\") pod \"cef9790b-68a2-4d22-8395-cbc48af58942\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " Jul 27 21:35:55.767183 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.767010 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cef9790b-68a2-4d22-8395-cbc48af58942-tls-certs\") pod \"cef9790b-68a2-4d22-8395-cbc48af58942\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " Jul 27 21:35:55.767183 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.767049 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-home\") pod \"cef9790b-68a2-4d22-8395-cbc48af58942\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " Jul 27 21:35:55.767183 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.767080 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2qlk\" (UniqueName: \"kubernetes.io/projected/cef9790b-68a2-4d22-8395-cbc48af58942-kube-api-access-m2qlk\") pod \"cef9790b-68a2-4d22-8395-cbc48af58942\" (UID: \"cef9790b-68a2-4d22-8395-cbc48af58942\") " Jul 27 21:35:55.767396 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.767199 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-model-cache" (OuterVolumeSpecName: "model-cache") pod "cef9790b-68a2-4d22-8395-cbc48af58942" (UID: "cef9790b-68a2-4d22-8395-cbc48af58942"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:35:55.767396 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.767214 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cef9790b-68a2-4d22-8395-cbc48af58942" (UID: "cef9790b-68a2-4d22-8395-cbc48af58942"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:35:55.767396 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.767345 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-home" (OuterVolumeSpecName: "home") pod "cef9790b-68a2-4d22-8395-cbc48af58942" (UID: "cef9790b-68a2-4d22-8395-cbc48af58942"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:35:55.767498 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.767402 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-model-cache\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:35:55.767498 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.767421 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-tmp-dir\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:35:55.769425 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.769395 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cef9790b-68a2-4d22-8395-cbc48af58942-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cef9790b-68a2-4d22-8395-cbc48af58942" (UID: "cef9790b-68a2-4d22-8395-cbc48af58942"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 21:35:55.769425 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.769414 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-dshm" (OuterVolumeSpecName: "dshm") pod "cef9790b-68a2-4d22-8395-cbc48af58942" (UID: "cef9790b-68a2-4d22-8395-cbc48af58942"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:35:55.769425 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.769413 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cef9790b-68a2-4d22-8395-cbc48af58942-kube-api-access-m2qlk" (OuterVolumeSpecName: "kube-api-access-m2qlk") pod "cef9790b-68a2-4d22-8395-cbc48af58942" (UID: "cef9790b-68a2-4d22-8395-cbc48af58942"). InnerVolumeSpecName "kube-api-access-m2qlk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:35:55.868317 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.868288 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cef9790b-68a2-4d22-8395-cbc48af58942-tls-certs\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:35:55.868317 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.868314 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-home\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:35:55.868317 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.868323 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2qlk\" (UniqueName: \"kubernetes.io/projected/cef9790b-68a2-4d22-8395-cbc48af58942-kube-api-access-m2qlk\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:35:55.868317 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:55.868332 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cef9790b-68a2-4d22-8395-cbc48af58942-dshm\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:35:56.168440 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:56.168407 2565 generic.go:356] "Generic (PLEG): container finished" podID="cef9790b-68a2-4d22-8395-cbc48af58942" containerID="10e47f4574442a6b4fac74ee894de51d583750e4fe5369f531d1b78f3b58a6e7" exitCode=0 Jul 27 21:35:56.168440 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:56.168444 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" event={"ID":"cef9790b-68a2-4d22-8395-cbc48af58942","Type":"ContainerDied","Data":"10e47f4574442a6b4fac74ee894de51d583750e4fe5369f531d1b78f3b58a6e7"} Jul 27 21:35:56.168689 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:56.168463 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" Jul 27 21:35:56.168689 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:56.168484 2565 scope.go:117] "RemoveContainer" containerID="10e47f4574442a6b4fac74ee894de51d583750e4fe5369f531d1b78f3b58a6e7" Jul 27 21:35:56.168689 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:56.168471 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9" event={"ID":"cef9790b-68a2-4d22-8395-cbc48af58942","Type":"ContainerDied","Data":"0c2ee2e1fabf79efabd6cd8d58c0ba7dbf5655ac8106bd4c24bbbe78d7ebb7da"} Jul 27 21:35:56.179685 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:56.179662 2565 scope.go:117] "RemoveContainer" containerID="10e47f4574442a6b4fac74ee894de51d583750e4fe5369f531d1b78f3b58a6e7" Jul 27 21:35:56.179995 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:35:56.179970 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10e47f4574442a6b4fac74ee894de51d583750e4fe5369f531d1b78f3b58a6e7\": container with ID starting with 10e47f4574442a6b4fac74ee894de51d583750e4fe5369f531d1b78f3b58a6e7 not found: ID does not exist" containerID="10e47f4574442a6b4fac74ee894de51d583750e4fe5369f531d1b78f3b58a6e7" Jul 27 21:35:56.180060 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:56.180007 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10e47f4574442a6b4fac74ee894de51d583750e4fe5369f531d1b78f3b58a6e7"} err="failed to get container status \"10e47f4574442a6b4fac74ee894de51d583750e4fe5369f531d1b78f3b58a6e7\": rpc error: code = NotFound desc = could not find container \"10e47f4574442a6b4fac74ee894de51d583750e4fe5369f531d1b78f3b58a6e7\": container with ID starting with 10e47f4574442a6b4fac74ee894de51d583750e4fe5369f531d1b78f3b58a6e7 not found: ID does not exist" Jul 27 21:35:56.193549 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:56.193526 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9"] Jul 27 21:35:56.200331 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:56.200307 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-85f004/auth-test-model-85f004-kserve-6b5d59f75-2bqw9"] Jul 27 21:35:58.053099 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:35:58.053068 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cef9790b-68a2-4d22-8395-cbc48af58942" path="/var/lib/kubelet/pods/cef9790b-68a2-4d22-8395-cbc48af58942/volumes" Jul 27 21:37:26.441136 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.441093 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4"] Jul 27 21:37:26.441620 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.441463 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cef9790b-68a2-4d22-8395-cbc48af58942" containerName="main" Jul 27 21:37:26.441620 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.441474 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="cef9790b-68a2-4d22-8395-cbc48af58942" containerName="main" Jul 27 21:37:26.441620 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.441558 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="cef9790b-68a2-4d22-8395-cbc48af58942" containerName="main" Jul 27 21:37:26.443802 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.443786 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.448416 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.448391 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c910ce\"/\"kube-root-ca.crt\"" Jul 27 21:37:26.448578 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.448448 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c910ce\"/\"openshift-service-ca.crt\"" Jul 27 21:37:26.448578 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.448460 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c910ce\"/\"rate-test-model-c910ce-kserve-self-signed-certs\"" Jul 27 21:37:26.448578 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.448494 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c910ce\"/\"default-dockercfg-k6k7w\"" Jul 27 21:37:26.456453 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.456429 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4"] Jul 27 21:37:26.470550 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.470520 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7b9q\" (UniqueName: \"kubernetes.io/projected/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-kube-api-access-w7b9q\") pod \"rate-test-model-c910ce-kserve-67df479975-ct7z4\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.470696 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.470560 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-home\") pod \"rate-test-model-c910ce-kserve-67df479975-ct7z4\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.470696 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.470641 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-dshm\") pod \"rate-test-model-c910ce-kserve-67df479975-ct7z4\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.470833 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.470694 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-tls-certs\") pod \"rate-test-model-c910ce-kserve-67df479975-ct7z4\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.470833 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.470721 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-tmp-dir\") pod \"rate-test-model-c910ce-kserve-67df479975-ct7z4\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.470833 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.470749 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-model-cache\") pod \"rate-test-model-c910ce-kserve-67df479975-ct7z4\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.572203 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.572161 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-dshm\") pod \"rate-test-model-c910ce-kserve-67df479975-ct7z4\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.572405 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.572227 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-tls-certs\") pod \"rate-test-model-c910ce-kserve-67df479975-ct7z4\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.572405 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.572287 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-tmp-dir\") pod \"rate-test-model-c910ce-kserve-67df479975-ct7z4\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.572405 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.572312 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-model-cache\") pod \"rate-test-model-c910ce-kserve-67df479975-ct7z4\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.572405 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.572398 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7b9q\" (UniqueName: \"kubernetes.io/projected/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-kube-api-access-w7b9q\") pod \"rate-test-model-c910ce-kserve-67df479975-ct7z4\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.572618 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.572425 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-home\") pod \"rate-test-model-c910ce-kserve-67df479975-ct7z4\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.572806 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.572776 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-tmp-dir\") pod \"rate-test-model-c910ce-kserve-67df479975-ct7z4\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.572806 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.572789 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-model-cache\") pod \"rate-test-model-c910ce-kserve-67df479975-ct7z4\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.573072 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.572852 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-home\") pod \"rate-test-model-c910ce-kserve-67df479975-ct7z4\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.574744 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.574717 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-dshm\") pod \"rate-test-model-c910ce-kserve-67df479975-ct7z4\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.574856 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.574840 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-tls-certs\") pod \"rate-test-model-c910ce-kserve-67df479975-ct7z4\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.580889 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.580866 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7b9q\" (UniqueName: \"kubernetes.io/projected/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-kube-api-access-w7b9q\") pod \"rate-test-model-c910ce-kserve-67df479975-ct7z4\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.752617 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.752534 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:26.887870 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:37:26.887826 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0cbca3be_63da_4f74_88e7_c5fdbf59d8a1.slice/crio-d72bf27271da02139f3dcaa0571a7d2b6a0be17a93f2dfaeb6ec38434de3ca93 WatchSource:0}: Error finding container d72bf27271da02139f3dcaa0571a7d2b6a0be17a93f2dfaeb6ec38434de3ca93: Status 404 returned error can't find the container with id d72bf27271da02139f3dcaa0571a7d2b6a0be17a93f2dfaeb6ec38434de3ca93 Jul 27 21:37:26.888051 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:26.888029 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4"] Jul 27 21:37:27.622594 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:27.622553 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" event={"ID":"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1","Type":"ContainerStarted","Data":"a1c0cac8b89c04ae248a3d9f9007a5448920bf47ce283bf3cfdeeda41e831256"} Jul 27 21:37:27.622928 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:27.622599 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" event={"ID":"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1","Type":"ContainerStarted","Data":"d72bf27271da02139f3dcaa0571a7d2b6a0be17a93f2dfaeb6ec38434de3ca93"} Jul 27 21:37:27.642943 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:27.642896 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" podStartSLOduration=1.642883638 podStartE2EDuration="1.642883638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:37:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:37:27.640703191 +0000 UTC m=+2776.163820913" watchObservedRunningTime="2026-07-27 21:37:27.642883638 +0000 UTC m=+2776.166001359" Jul 27 21:37:36.752860 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:36.752779 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:36.752860 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:36.752834 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:36.765300 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:36.765225 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:37:37.680669 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:37:37.680643 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:39:30.850960 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:30.850876 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4"] Jul 27 21:39:45.871651 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:45.871603 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" podUID="0cbca3be-63da-4f74-88e7-c5fdbf59d8a1" containerName="main" containerID="cri-o://a1c0cac8b89c04ae248a3d9f9007a5448920bf47ce283bf3cfdeeda41e831256" gracePeriod=45 Jul 27 21:39:46.126922 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.126859 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:39:46.147615 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.147585 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-tls-certs\") pod \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " Jul 27 21:39:46.147788 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.147652 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-tmp-dir\") pod \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " Jul 27 21:39:46.147788 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.147693 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-home\") pod \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " Jul 27 21:39:46.147788 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.147731 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-dshm\") pod \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " Jul 27 21:39:46.147788 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.147763 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7b9q\" (UniqueName: \"kubernetes.io/projected/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-kube-api-access-w7b9q\") pod \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " Jul 27 21:39:46.148001 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.147821 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-model-cache\") pod \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\" (UID: \"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1\") " Jul 27 21:39:46.148001 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.147929 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-home" (OuterVolumeSpecName: "home") pod "0cbca3be-63da-4f74-88e7-c5fdbf59d8a1" (UID: "0cbca3be-63da-4f74-88e7-c5fdbf59d8a1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:39:46.148435 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.148142 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-home\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:39:46.148435 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.148354 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0cbca3be-63da-4f74-88e7-c5fdbf59d8a1" (UID: "0cbca3be-63da-4f74-88e7-c5fdbf59d8a1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:39:46.148435 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.148370 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-model-cache" (OuterVolumeSpecName: "model-cache") pod "0cbca3be-63da-4f74-88e7-c5fdbf59d8a1" (UID: "0cbca3be-63da-4f74-88e7-c5fdbf59d8a1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:39:46.150422 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.150395 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-dshm" (OuterVolumeSpecName: "dshm") pod "0cbca3be-63da-4f74-88e7-c5fdbf59d8a1" (UID: "0cbca3be-63da-4f74-88e7-c5fdbf59d8a1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:39:46.150674 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.150654 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0cbca3be-63da-4f74-88e7-c5fdbf59d8a1" (UID: "0cbca3be-63da-4f74-88e7-c5fdbf59d8a1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 21:39:46.150674 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.150660 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-kube-api-access-w7b9q" (OuterVolumeSpecName: "kube-api-access-w7b9q") pod "0cbca3be-63da-4f74-88e7-c5fdbf59d8a1" (UID: "0cbca3be-63da-4f74-88e7-c5fdbf59d8a1"). InnerVolumeSpecName "kube-api-access-w7b9q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:39:46.249337 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.249299 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-tls-certs\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:39:46.249337 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.249329 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-tmp-dir\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:39:46.249337 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.249338 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-dshm\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:39:46.249337 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.249346 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7b9q\" (UniqueName: \"kubernetes.io/projected/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-kube-api-access-w7b9q\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:39:46.249609 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.249356 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1-model-cache\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:39:46.322103 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.322065 2565 generic.go:356] "Generic (PLEG): container finished" podID="0cbca3be-63da-4f74-88e7-c5fdbf59d8a1" containerID="a1c0cac8b89c04ae248a3d9f9007a5448920bf47ce283bf3cfdeeda41e831256" exitCode=0 Jul 27 21:39:46.322340 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.322119 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" Jul 27 21:39:46.322340 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.322155 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" event={"ID":"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1","Type":"ContainerDied","Data":"a1c0cac8b89c04ae248a3d9f9007a5448920bf47ce283bf3cfdeeda41e831256"} Jul 27 21:39:46.322340 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.322198 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4" event={"ID":"0cbca3be-63da-4f74-88e7-c5fdbf59d8a1","Type":"ContainerDied","Data":"d72bf27271da02139f3dcaa0571a7d2b6a0be17a93f2dfaeb6ec38434de3ca93"} Jul 27 21:39:46.322340 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.322215 2565 scope.go:117] "RemoveContainer" containerID="a1c0cac8b89c04ae248a3d9f9007a5448920bf47ce283bf3cfdeeda41e831256" Jul 27 21:39:46.331796 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.331778 2565 scope.go:117] "RemoveContainer" containerID="a1c0cac8b89c04ae248a3d9f9007a5448920bf47ce283bf3cfdeeda41e831256" Jul 27 21:39:46.332057 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:39:46.332038 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1c0cac8b89c04ae248a3d9f9007a5448920bf47ce283bf3cfdeeda41e831256\": container with ID starting with a1c0cac8b89c04ae248a3d9f9007a5448920bf47ce283bf3cfdeeda41e831256 not found: ID does not exist" containerID="a1c0cac8b89c04ae248a3d9f9007a5448920bf47ce283bf3cfdeeda41e831256" Jul 27 21:39:46.332101 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.332066 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1c0cac8b89c04ae248a3d9f9007a5448920bf47ce283bf3cfdeeda41e831256"} err="failed to get container status \"a1c0cac8b89c04ae248a3d9f9007a5448920bf47ce283bf3cfdeeda41e831256\": rpc error: code = NotFound desc = could not find container \"a1c0cac8b89c04ae248a3d9f9007a5448920bf47ce283bf3cfdeeda41e831256\": container with ID starting with a1c0cac8b89c04ae248a3d9f9007a5448920bf47ce283bf3cfdeeda41e831256 not found: ID does not exist" Jul 27 21:39:46.354196 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.354161 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4"] Jul 27 21:39:46.372995 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:46.372962 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c910ce/rate-test-model-c910ce-kserve-67df479975-ct7z4"] Jul 27 21:39:48.052135 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:48.052097 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cbca3be-63da-4f74-88e7-c5fdbf59d8a1" path="/var/lib/kubelet/pods/0cbca3be-63da-4f74-88e7-c5fdbf59d8a1/volumes" Jul 27 21:39:50.837026 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.836993 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86"] Jul 27 21:39:50.837518 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.837377 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cbca3be-63da-4f74-88e7-c5fdbf59d8a1" containerName="main" Jul 27 21:39:50.837518 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.837390 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cbca3be-63da-4f74-88e7-c5fdbf59d8a1" containerName="main" Jul 27 21:39:50.837518 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.837471 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cbca3be-63da-4f74-88e7-c5fdbf59d8a1" containerName="main" Jul 27 21:39:50.839440 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.839425 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.842634 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.842607 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-5d8041-openshift-default-dockercfg-sglpd\"" Jul 27 21:39:50.857435 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.857409 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86"] Jul 27 21:39:50.894947 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.894913 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-workload-certs\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.894947 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.894952 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-istio-envoy\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.895199 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.894973 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-istio-data\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.895199 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.895002 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abaafb18-9795-413a-a48e-381897e863f3-istiod-ca-cert\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.895199 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.895030 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd4hh\" (UniqueName: \"kubernetes.io/projected/abaafb18-9795-413a-a48e-381897e863f3-kube-api-access-gd4hh\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.895199 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.895064 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abaafb18-9795-413a-a48e-381897e863f3-istio-podinfo\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.895199 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.895106 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-credential-socket\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.895199 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.895131 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-workload-socket\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.895199 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.895188 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abaafb18-9795-413a-a48e-381897e863f3-istio-token\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.996514 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.996472 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abaafb18-9795-413a-a48e-381897e863f3-istio-podinfo\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.996717 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.996521 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-credential-socket\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.996717 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.996559 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-workload-socket\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.996717 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.996590 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abaafb18-9795-413a-a48e-381897e863f3-istio-token\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.996717 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.996631 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-workload-certs\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.996717 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.996663 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-istio-envoy\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.996717 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.996688 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-istio-data\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.997094 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.996725 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abaafb18-9795-413a-a48e-381897e863f3-istiod-ca-cert\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.997094 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.996760 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd4hh\" (UniqueName: \"kubernetes.io/projected/abaafb18-9795-413a-a48e-381897e863f3-kube-api-access-gd4hh\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.997094 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.997042 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-credential-socket\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.997094 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.997067 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-workload-socket\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.997382 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.997139 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-workload-certs\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.997528 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.997439 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-istio-data\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.997724 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.997704 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abaafb18-9795-413a-a48e-381897e863f3-istiod-ca-cert\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.999091 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.999067 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-istio-envoy\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:50.999320 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:50.999302 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abaafb18-9795-413a-a48e-381897e863f3-istio-podinfo\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:51.005846 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:51.005816 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abaafb18-9795-413a-a48e-381897e863f3-istio-token\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:51.005948 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:51.005922 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd4hh\" (UniqueName: \"kubernetes.io/projected/abaafb18-9795-413a-a48e-381897e863f3-kube-api-access-gd4hh\") pod \"e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:51.149299 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:51.149151 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:51.293815 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:51.293788 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86"] Jul 27 21:39:51.295269 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:39:51.295224 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabaafb18_9795_413a_a48e_381897e863f3.slice/crio-d41ecc97fcef6a4c7d4e2576e14cef98bb4a1cdaac7506f44982cf490c86e1f2 WatchSource:0}: Error finding container d41ecc97fcef6a4c7d4e2576e14cef98bb4a1cdaac7506f44982cf490c86e1f2: Status 404 returned error can't find the container with id d41ecc97fcef6a4c7d4e2576e14cef98bb4a1cdaac7506f44982cf490c86e1f2 Jul 27 21:39:51.297516 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:51.297478 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:39:51.297604 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:51.297556 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:39:51.297604 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:51.297588 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:39:51.355107 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:51.355078 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" event={"ID":"abaafb18-9795-413a-a48e-381897e863f3","Type":"ContainerStarted","Data":"d41ecc97fcef6a4c7d4e2576e14cef98bb4a1cdaac7506f44982cf490c86e1f2"} Jul 27 21:39:52.362688 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:52.362648 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" event={"ID":"abaafb18-9795-413a-a48e-381897e863f3","Type":"ContainerStarted","Data":"39f3e49fb6c9cbd770d198d979f165afa35c0053eb6f5b5e4804f42cea333929"} Jul 27 21:39:52.383448 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:52.383392 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" podStartSLOduration=2.383375535 podStartE2EDuration="2.383375535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:39:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:39:52.382051182 +0000 UTC m=+2920.905168906" watchObservedRunningTime="2026-07-27 21:39:52.383375535 +0000 UTC m=+2920.906493254" Jul 27 21:39:53.149830 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:53.149792 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:53.154915 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:53.154889 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:53.367724 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:53.367689 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:39:53.368718 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:39:53.368697 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:40:09.410468 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:09.410420 2565 patch_prober.go:28] interesting pod/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 27 21:40:09.410880 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:09.410498 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" podUID="abaafb18-9795-413a-a48e-381897e863f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 27 21:40:11.469993 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:11.469957 2565 generic.go:356] "Generic (PLEG): container finished" podID="abaafb18-9795-413a-a48e-381897e863f3" containerID="39f3e49fb6c9cbd770d198d979f165afa35c0053eb6f5b5e4804f42cea333929" exitCode=0 Jul 27 21:40:11.470428 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:11.470031 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" event={"ID":"abaafb18-9795-413a-a48e-381897e863f3","Type":"ContainerDied","Data":"39f3e49fb6c9cbd770d198d979f165afa35c0053eb6f5b5e4804f42cea333929"} Jul 27 21:40:11.470485 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:11.470456 2565 scope.go:117] "RemoveContainer" containerID="39f3e49fb6c9cbd770d198d979f165afa35c0053eb6f5b5e4804f42cea333929" Jul 27 21:40:11.471511 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:11.471482 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:40:11.471580 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:11.471546 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:40:11.471615 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:11.471579 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:40:12.481112 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:12.481060 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" event={"ID":"abaafb18-9795-413a-a48e-381897e863f3","Type":"ContainerStarted","Data":"d45b45b07c6d81dd9a6f5cf0b04c14f77748bc556c233af0980e1ab204b7735a"} Jul 27 21:40:13.149721 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:13.149675 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:40:13.149942 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:13.149906 2565 patch_prober.go:28] interesting pod/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 27 21:40:13.150019 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:13.149982 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" podUID="abaafb18-9795-413a-a48e-381897e863f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 27 21:40:14.150165 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:14.150123 2565 patch_prober.go:28] interesting pod/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 27 21:40:14.150687 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:14.150182 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" podUID="abaafb18-9795-413a-a48e-381897e863f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 27 21:40:15.150089 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:15.150052 2565 patch_prober.go:28] interesting pod/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 27 21:40:15.150320 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:15.150107 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" podUID="abaafb18-9795-413a-a48e-381897e863f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 27 21:40:16.150105 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:16.150065 2565 patch_prober.go:28] interesting pod/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 27 21:40:16.150321 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:16.150123 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" podUID="abaafb18-9795-413a-a48e-381897e863f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 27 21:40:17.153643 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:17.153604 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:40:17.154044 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:17.153921 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:40:17.154892 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:40:17.154871 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:41:36.792965 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:36.792930 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84"] Jul 27 21:41:36.801596 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:36.801570 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:36.804849 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:36.804824 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-ef0509-openshift-default-dockercfg-s4sl2\"" Jul 27 21:41:36.814062 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:36.814043 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84"] Jul 27 21:41:36.902501 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:36.902465 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-credential-socket\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:36.902881 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:36.902843 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6rmd\" (UniqueName: \"kubernetes.io/projected/4dbda4df-3439-4e59-a37c-1dd761819778-kube-api-access-c6rmd\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:36.903080 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:36.903055 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dbda4df-3439-4e59-a37c-1dd761819778-istiod-ca-cert\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:36.903268 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:36.903102 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dbda4df-3439-4e59-a37c-1dd761819778-istio-podinfo\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:36.903268 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:36.903139 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-workload-certs\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:36.903441 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:36.903301 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-istio-envoy\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:36.903441 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:36.903361 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-workload-socket\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:36.903441 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:36.903395 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dbda4df-3439-4e59-a37c-1dd761819778-istio-token\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:36.903655 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:36.903452 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-istio-data\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.004729 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.004695 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dbda4df-3439-4e59-a37c-1dd761819778-istiod-ca-cert\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.004930 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.004741 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dbda4df-3439-4e59-a37c-1dd761819778-istio-podinfo\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.004930 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.004776 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-workload-certs\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.004930 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.004844 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-istio-envoy\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.004930 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.004891 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-workload-socket\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.005155 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.004931 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dbda4df-3439-4e59-a37c-1dd761819778-istio-token\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.005155 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.004998 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-istio-data\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.005155 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.005032 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-credential-socket\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.005155 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.005056 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6rmd\" (UniqueName: \"kubernetes.io/projected/4dbda4df-3439-4e59-a37c-1dd761819778-kube-api-access-c6rmd\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.005432 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.005401 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-workload-socket\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.005432 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.005415 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-istio-data\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.005586 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.005559 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dbda4df-3439-4e59-a37c-1dd761819778-istiod-ca-cert\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.005716 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.005585 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-credential-socket\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.005775 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.005620 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-workload-certs\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.007398 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.007380 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-istio-envoy\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.007458 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.007426 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dbda4df-3439-4e59-a37c-1dd761819778-istio-podinfo\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.015872 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.015834 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6rmd\" (UniqueName: \"kubernetes.io/projected/4dbda4df-3439-4e59-a37c-1dd761819778-kube-api-access-c6rmd\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.016387 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.016370 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dbda4df-3439-4e59-a37c-1dd761819778-istio-token\") pod \"e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.112835 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.112753 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:37.299697 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.299672 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84"] Jul 27 21:41:37.301313 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:41:37.301285 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dbda4df_3439_4e59_a37c_1dd761819778.slice/crio-08252f383da7d0d180be9a9f3016f0be4612a7af1ff9874f9f9d9a526b7635a4 WatchSource:0}: Error finding container 08252f383da7d0d180be9a9f3016f0be4612a7af1ff9874f9f9d9a526b7635a4: Status 404 returned error can't find the container with id 08252f383da7d0d180be9a9f3016f0be4612a7af1ff9874f9f9d9a526b7635a4 Jul 27 21:41:37.303525 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.303488 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:41:37.303627 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.303574 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:41:37.303687 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.303646 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:41:37.923739 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.923699 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" event={"ID":"4dbda4df-3439-4e59-a37c-1dd761819778","Type":"ContainerStarted","Data":"846e76baf69e7f8120aa71104dccfe29ce110cd57013e548db75c52e11568863"} Jul 27 21:41:37.924278 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.923760 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" event={"ID":"4dbda4df-3439-4e59-a37c-1dd761819778","Type":"ContainerStarted","Data":"08252f383da7d0d180be9a9f3016f0be4612a7af1ff9874f9f9d9a526b7635a4"} Jul 27 21:41:37.947466 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:37.947415 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" podStartSLOduration=1.947397938 podStartE2EDuration="1.947397938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:41:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:41:37.945111119 +0000 UTC m=+3026.468228839" watchObservedRunningTime="2026-07-27 21:41:37.947397938 +0000 UTC m=+3026.470515752" Jul 27 21:41:38.113098 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:38.113063 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:38.118096 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:38.118067 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:38.930081 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:38.930048 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:38.930959 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:38.930942 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:41:59.034682 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:59.034597 2565 generic.go:356] "Generic (PLEG): container finished" podID="4dbda4df-3439-4e59-a37c-1dd761819778" containerID="846e76baf69e7f8120aa71104dccfe29ce110cd57013e548db75c52e11568863" exitCode=0 Jul 27 21:41:59.034682 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:59.034666 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" event={"ID":"4dbda4df-3439-4e59-a37c-1dd761819778","Type":"ContainerDied","Data":"846e76baf69e7f8120aa71104dccfe29ce110cd57013e548db75c52e11568863"} Jul 27 21:41:59.035104 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:59.035076 2565 scope.go:117] "RemoveContainer" containerID="846e76baf69e7f8120aa71104dccfe29ce110cd57013e548db75c52e11568863" Jul 27 21:41:59.036279 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:59.036229 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:41:59.036374 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:59.036320 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:41:59.036374 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:41:59.036353 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 27 21:42:00.041591 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:00.041553 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" event={"ID":"4dbda4df-3439-4e59-a37c-1dd761819778","Type":"ContainerStarted","Data":"c7ee4525233c0cfc87491801f4d5881ec14950d1733f010763fae009d678d19a"} Jul 27 21:42:00.113876 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:00.113839 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:42:00.114049 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:00.114020 2565 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 27 21:42:00.114091 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:00.114067 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" podUID="4dbda4df-3439-4e59-a37c-1dd761819778" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 27 21:42:01.113191 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:01.113154 2565 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 27 21:42:01.113619 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:01.113224 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" podUID="4dbda4df-3439-4e59-a37c-1dd761819778" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 27 21:42:02.113610 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:02.113572 2565 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 27 21:42:02.113999 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:02.113624 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" podUID="4dbda4df-3439-4e59-a37c-1dd761819778" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 27 21:42:03.114093 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:03.114048 2565 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 27 21:42:03.114610 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:03.114112 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" podUID="4dbda4df-3439-4e59-a37c-1dd761819778" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 27 21:42:04.117202 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:04.117172 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:42:04.117647 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:04.117628 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:42:04.118313 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:04.118293 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:42:22.236003 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:22.235971 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84"] Jul 27 21:42:22.236611 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:22.236174 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" podUID="4dbda4df-3439-4e59-a37c-1dd761819778" containerName="istio-proxy" containerID="cri-o://c7ee4525233c0cfc87491801f4d5881ec14950d1733f010763fae009d678d19a" gracePeriod=30 Jul 27 21:42:27.485585 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.485561 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:42:27.577249 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.577150 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-credential-socket\") pod \"4dbda4df-3439-4e59-a37c-1dd761819778\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " Jul 27 21:42:27.577249 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.577186 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-workload-certs\") pod \"4dbda4df-3439-4e59-a37c-1dd761819778\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " Jul 27 21:42:27.577249 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.577220 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-istio-envoy\") pod \"4dbda4df-3439-4e59-a37c-1dd761819778\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " Jul 27 21:42:27.577516 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.577260 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-istio-data\") pod \"4dbda4df-3439-4e59-a37c-1dd761819778\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " Jul 27 21:42:27.577516 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.577280 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-workload-socket\") pod \"4dbda4df-3439-4e59-a37c-1dd761819778\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " Jul 27 21:42:27.577516 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.577311 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6rmd\" (UniqueName: \"kubernetes.io/projected/4dbda4df-3439-4e59-a37c-1dd761819778-kube-api-access-c6rmd\") pod \"4dbda4df-3439-4e59-a37c-1dd761819778\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " Jul 27 21:42:27.577516 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.577367 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dbda4df-3439-4e59-a37c-1dd761819778-istio-token\") pod \"4dbda4df-3439-4e59-a37c-1dd761819778\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " Jul 27 21:42:27.577516 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.577395 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dbda4df-3439-4e59-a37c-1dd761819778-istiod-ca-cert\") pod \"4dbda4df-3439-4e59-a37c-1dd761819778\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " Jul 27 21:42:27.577516 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.577424 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dbda4df-3439-4e59-a37c-1dd761819778-istio-podinfo\") pod \"4dbda4df-3439-4e59-a37c-1dd761819778\" (UID: \"4dbda4df-3439-4e59-a37c-1dd761819778\") " Jul 27 21:42:27.577516 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.577442 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4dbda4df-3439-4e59-a37c-1dd761819778" (UID: "4dbda4df-3439-4e59-a37c-1dd761819778"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:42:27.577516 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.577483 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4dbda4df-3439-4e59-a37c-1dd761819778" (UID: "4dbda4df-3439-4e59-a37c-1dd761819778"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:42:27.577939 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.577598 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4dbda4df-3439-4e59-a37c-1dd761819778" (UID: "4dbda4df-3439-4e59-a37c-1dd761819778"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:42:27.577939 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.577778 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-workload-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:42:27.577939 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.577796 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-credential-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:42:27.577939 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.577820 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-workload-certs\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:42:27.577939 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.577863 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dbda4df-3439-4e59-a37c-1dd761819778-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4dbda4df-3439-4e59-a37c-1dd761819778" (UID: "4dbda4df-3439-4e59-a37c-1dd761819778"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 21:42:27.578492 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.578466 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-istio-data" (OuterVolumeSpecName: "istio-data") pod "4dbda4df-3439-4e59-a37c-1dd761819778" (UID: "4dbda4df-3439-4e59-a37c-1dd761819778"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:42:27.579771 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.579747 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4dbda4df-3439-4e59-a37c-1dd761819778-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4dbda4df-3439-4e59-a37c-1dd761819778" (UID: "4dbda4df-3439-4e59-a37c-1dd761819778"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 27 21:42:27.579881 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.579781 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dbda4df-3439-4e59-a37c-1dd761819778-istio-token" (OuterVolumeSpecName: "istio-token") pod "4dbda4df-3439-4e59-a37c-1dd761819778" (UID: "4dbda4df-3439-4e59-a37c-1dd761819778"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:42:27.580030 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.580001 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4dbda4df-3439-4e59-a37c-1dd761819778" (UID: "4dbda4df-3439-4e59-a37c-1dd761819778"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:42:27.580030 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.580011 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dbda4df-3439-4e59-a37c-1dd761819778-kube-api-access-c6rmd" (OuterVolumeSpecName: "kube-api-access-c6rmd") pod "4dbda4df-3439-4e59-a37c-1dd761819778" (UID: "4dbda4df-3439-4e59-a37c-1dd761819778"). InnerVolumeSpecName "kube-api-access-c6rmd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:42:27.678724 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.678688 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dbda4df-3439-4e59-a37c-1dd761819778-istio-podinfo\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:42:27.678724 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.678721 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-istio-envoy\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:42:27.678724 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.678732 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dbda4df-3439-4e59-a37c-1dd761819778-istio-data\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:42:27.678944 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.678740 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6rmd\" (UniqueName: \"kubernetes.io/projected/4dbda4df-3439-4e59-a37c-1dd761819778-kube-api-access-c6rmd\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:42:27.678944 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.678750 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dbda4df-3439-4e59-a37c-1dd761819778-istio-token\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:42:27.678944 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:27.678757 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dbda4df-3439-4e59-a37c-1dd761819778-istiod-ca-cert\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:42:28.184453 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:28.184416 2565 generic.go:356] "Generic (PLEG): container finished" podID="4dbda4df-3439-4e59-a37c-1dd761819778" containerID="c7ee4525233c0cfc87491801f4d5881ec14950d1733f010763fae009d678d19a" exitCode=0 Jul 27 21:42:28.184615 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:28.184463 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" Jul 27 21:42:28.184615 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:28.184502 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" event={"ID":"4dbda4df-3439-4e59-a37c-1dd761819778","Type":"ContainerDied","Data":"c7ee4525233c0cfc87491801f4d5881ec14950d1733f010763fae009d678d19a"} Jul 27 21:42:28.184615 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:28.184558 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84" event={"ID":"4dbda4df-3439-4e59-a37c-1dd761819778","Type":"ContainerDied","Data":"08252f383da7d0d180be9a9f3016f0be4612a7af1ff9874f9f9d9a526b7635a4"} Jul 27 21:42:28.184615 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:28.184581 2565 scope.go:117] "RemoveContainer" containerID="c7ee4525233c0cfc87491801f4d5881ec14950d1733f010763fae009d678d19a" Jul 27 21:42:28.193595 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:28.193575 2565 scope.go:117] "RemoveContainer" containerID="846e76baf69e7f8120aa71104dccfe29ce110cd57013e548db75c52e11568863" Jul 27 21:42:28.201139 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:28.201116 2565 scope.go:117] "RemoveContainer" containerID="c7ee4525233c0cfc87491801f4d5881ec14950d1733f010763fae009d678d19a" Jul 27 21:42:28.201775 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:42:28.201445 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7ee4525233c0cfc87491801f4d5881ec14950d1733f010763fae009d678d19a\": container with ID starting with c7ee4525233c0cfc87491801f4d5881ec14950d1733f010763fae009d678d19a not found: ID does not exist" containerID="c7ee4525233c0cfc87491801f4d5881ec14950d1733f010763fae009d678d19a" Jul 27 21:42:28.201775 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:28.201481 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7ee4525233c0cfc87491801f4d5881ec14950d1733f010763fae009d678d19a"} err="failed to get container status \"c7ee4525233c0cfc87491801f4d5881ec14950d1733f010763fae009d678d19a\": rpc error: code = NotFound desc = could not find container \"c7ee4525233c0cfc87491801f4d5881ec14950d1733f010763fae009d678d19a\": container with ID starting with c7ee4525233c0cfc87491801f4d5881ec14950d1733f010763fae009d678d19a not found: ID does not exist" Jul 27 21:42:28.201775 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:28.201504 2565 scope.go:117] "RemoveContainer" containerID="846e76baf69e7f8120aa71104dccfe29ce110cd57013e548db75c52e11568863" Jul 27 21:42:28.202041 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:42:28.201841 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"846e76baf69e7f8120aa71104dccfe29ce110cd57013e548db75c52e11568863\": container with ID starting with 846e76baf69e7f8120aa71104dccfe29ce110cd57013e548db75c52e11568863 not found: ID does not exist" containerID="846e76baf69e7f8120aa71104dccfe29ce110cd57013e548db75c52e11568863" Jul 27 21:42:28.202041 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:28.201873 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"846e76baf69e7f8120aa71104dccfe29ce110cd57013e548db75c52e11568863"} err="failed to get container status \"846e76baf69e7f8120aa71104dccfe29ce110cd57013e548db75c52e11568863\": rpc error: code = NotFound desc = could not find container \"846e76baf69e7f8120aa71104dccfe29ce110cd57013e548db75c52e11568863\": container with ID starting with 846e76baf69e7f8120aa71104dccfe29ce110cd57013e548db75c52e11568863 not found: ID does not exist" Jul 27 21:42:28.204498 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:28.204476 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84"] Jul 27 21:42:28.208226 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:28.208207 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-ef0509-openshift-default-5889677654-rgf84"] Jul 27 21:42:30.052670 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:42:30.052642 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dbda4df-3439-4e59-a37c-1dd761819778" path="/var/lib/kubelet/pods/4dbda4df-3439-4e59-a37c-1dd761819778/volumes" Jul 27 21:43:35.708578 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:35.708496 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86"] Jul 27 21:43:35.709149 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:35.708733 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" podUID="abaafb18-9795-413a-a48e-381897e863f3" containerName="istio-proxy" containerID="cri-o://d45b45b07c6d81dd9a6f5cf0b04c14f77748bc556c233af0980e1ab204b7735a" gracePeriod=30 Jul 27 21:43:40.960048 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:40.960021 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:43:41.056102 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.056024 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-workload-socket\") pod \"abaafb18-9795-413a-a48e-381897e863f3\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " Jul 27 21:43:41.056102 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.056058 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abaafb18-9795-413a-a48e-381897e863f3-istio-token\") pod \"abaafb18-9795-413a-a48e-381897e863f3\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " Jul 27 21:43:41.056347 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.056129 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-workload-certs\") pod \"abaafb18-9795-413a-a48e-381897e863f3\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " Jul 27 21:43:41.056347 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.056151 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-istio-data\") pod \"abaafb18-9795-413a-a48e-381897e863f3\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " Jul 27 21:43:41.056347 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.056184 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abaafb18-9795-413a-a48e-381897e863f3-istio-podinfo\") pod \"abaafb18-9795-413a-a48e-381897e863f3\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " Jul 27 21:43:41.056347 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.056283 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-istio-envoy\") pod \"abaafb18-9795-413a-a48e-381897e863f3\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " Jul 27 21:43:41.056347 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.056321 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abaafb18-9795-413a-a48e-381897e863f3-istiod-ca-cert\") pod \"abaafb18-9795-413a-a48e-381897e863f3\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " Jul 27 21:43:41.056347 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.056345 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-credential-socket\") pod \"abaafb18-9795-413a-a48e-381897e863f3\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " Jul 27 21:43:41.056639 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.056356 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "abaafb18-9795-413a-a48e-381897e863f3" (UID: "abaafb18-9795-413a-a48e-381897e863f3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:43:41.056639 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.056372 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gd4hh\" (UniqueName: \"kubernetes.io/projected/abaafb18-9795-413a-a48e-381897e863f3-kube-api-access-gd4hh\") pod \"abaafb18-9795-413a-a48e-381897e863f3\" (UID: \"abaafb18-9795-413a-a48e-381897e863f3\") " Jul 27 21:43:41.056639 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.056477 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "abaafb18-9795-413a-a48e-381897e863f3" (UID: "abaafb18-9795-413a-a48e-381897e863f3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:43:41.056784 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.056687 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abaafb18-9795-413a-a48e-381897e863f3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "abaafb18-9795-413a-a48e-381897e863f3" (UID: "abaafb18-9795-413a-a48e-381897e863f3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 21:43:41.056833 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.056788 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-workload-certs\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:43:41.056833 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.056804 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abaafb18-9795-413a-a48e-381897e863f3-istiod-ca-cert\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:43:41.056833 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.056816 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-workload-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:43:41.056982 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.056893 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "abaafb18-9795-413a-a48e-381897e863f3" (UID: "abaafb18-9795-413a-a48e-381897e863f3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:43:41.057222 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.057198 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-istio-data" (OuterVolumeSpecName: "istio-data") pod "abaafb18-9795-413a-a48e-381897e863f3" (UID: "abaafb18-9795-413a-a48e-381897e863f3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:43:41.058551 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.058524 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "abaafb18-9795-413a-a48e-381897e863f3" (UID: "abaafb18-9795-413a-a48e-381897e863f3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:43:41.058675 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.058597 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abaafb18-9795-413a-a48e-381897e863f3-kube-api-access-gd4hh" (OuterVolumeSpecName: "kube-api-access-gd4hh") pod "abaafb18-9795-413a-a48e-381897e863f3" (UID: "abaafb18-9795-413a-a48e-381897e863f3"). InnerVolumeSpecName "kube-api-access-gd4hh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:43:41.058800 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.058785 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abaafb18-9795-413a-a48e-381897e863f3-istio-token" (OuterVolumeSpecName: "istio-token") pod "abaafb18-9795-413a-a48e-381897e863f3" (UID: "abaafb18-9795-413a-a48e-381897e863f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:43:41.058867 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.058847 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/abaafb18-9795-413a-a48e-381897e863f3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "abaafb18-9795-413a-a48e-381897e863f3" (UID: "abaafb18-9795-413a-a48e-381897e863f3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 27 21:43:41.157572 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.157547 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-istio-envoy\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:43:41.157572 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.157571 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-credential-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:43:41.157572 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.157581 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gd4hh\" (UniqueName: \"kubernetes.io/projected/abaafb18-9795-413a-a48e-381897e863f3-kube-api-access-gd4hh\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:43:41.157846 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.157602 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abaafb18-9795-413a-a48e-381897e863f3-istio-token\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:43:41.157846 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.157610 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abaafb18-9795-413a-a48e-381897e863f3-istio-data\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:43:41.157846 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.157617 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abaafb18-9795-413a-a48e-381897e863f3-istio-podinfo\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:43:41.557933 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.557894 2565 generic.go:356] "Generic (PLEG): container finished" podID="abaafb18-9795-413a-a48e-381897e863f3" containerID="d45b45b07c6d81dd9a6f5cf0b04c14f77748bc556c233af0980e1ab204b7735a" exitCode=0 Jul 27 21:43:41.558131 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.557958 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" Jul 27 21:43:41.558131 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.557976 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" event={"ID":"abaafb18-9795-413a-a48e-381897e863f3","Type":"ContainerDied","Data":"d45b45b07c6d81dd9a6f5cf0b04c14f77748bc556c233af0980e1ab204b7735a"} Jul 27 21:43:41.558131 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.558057 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86" event={"ID":"abaafb18-9795-413a-a48e-381897e863f3","Type":"ContainerDied","Data":"d41ecc97fcef6a4c7d4e2576e14cef98bb4a1cdaac7506f44982cf490c86e1f2"} Jul 27 21:43:41.558131 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.558087 2565 scope.go:117] "RemoveContainer" containerID="d45b45b07c6d81dd9a6f5cf0b04c14f77748bc556c233af0980e1ab204b7735a" Jul 27 21:43:41.567778 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.567753 2565 scope.go:117] "RemoveContainer" containerID="39f3e49fb6c9cbd770d198d979f165afa35c0053eb6f5b5e4804f42cea333929" Jul 27 21:43:41.575738 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.575718 2565 scope.go:117] "RemoveContainer" containerID="d45b45b07c6d81dd9a6f5cf0b04c14f77748bc556c233af0980e1ab204b7735a" Jul 27 21:43:41.575970 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:43:41.575951 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d45b45b07c6d81dd9a6f5cf0b04c14f77748bc556c233af0980e1ab204b7735a\": container with ID starting with d45b45b07c6d81dd9a6f5cf0b04c14f77748bc556c233af0980e1ab204b7735a not found: ID does not exist" containerID="d45b45b07c6d81dd9a6f5cf0b04c14f77748bc556c233af0980e1ab204b7735a" Jul 27 21:43:41.576019 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.575979 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d45b45b07c6d81dd9a6f5cf0b04c14f77748bc556c233af0980e1ab204b7735a"} err="failed to get container status \"d45b45b07c6d81dd9a6f5cf0b04c14f77748bc556c233af0980e1ab204b7735a\": rpc error: code = NotFound desc = could not find container \"d45b45b07c6d81dd9a6f5cf0b04c14f77748bc556c233af0980e1ab204b7735a\": container with ID starting with d45b45b07c6d81dd9a6f5cf0b04c14f77748bc556c233af0980e1ab204b7735a not found: ID does not exist" Jul 27 21:43:41.576019 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.575997 2565 scope.go:117] "RemoveContainer" containerID="39f3e49fb6c9cbd770d198d979f165afa35c0053eb6f5b5e4804f42cea333929" Jul 27 21:43:41.576193 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:43:41.576180 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39f3e49fb6c9cbd770d198d979f165afa35c0053eb6f5b5e4804f42cea333929\": container with ID starting with 39f3e49fb6c9cbd770d198d979f165afa35c0053eb6f5b5e4804f42cea333929 not found: ID does not exist" containerID="39f3e49fb6c9cbd770d198d979f165afa35c0053eb6f5b5e4804f42cea333929" Jul 27 21:43:41.576257 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.576196 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39f3e49fb6c9cbd770d198d979f165afa35c0053eb6f5b5e4804f42cea333929"} err="failed to get container status \"39f3e49fb6c9cbd770d198d979f165afa35c0053eb6f5b5e4804f42cea333929\": rpc error: code = NotFound desc = could not find container \"39f3e49fb6c9cbd770d198d979f165afa35c0053eb6f5b5e4804f42cea333929\": container with ID starting with 39f3e49fb6c9cbd770d198d979f165afa35c0053eb6f5b5e4804f42cea333929 not found: ID does not exist" Jul 27 21:43:41.579861 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.579841 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86"] Jul 27 21:43:41.582590 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:41.582572 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-5d8041-openshift-default-d6558dd4b-n5b86"] Jul 27 21:43:42.052965 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:43:42.052934 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abaafb18-9795-413a-a48e-381897e863f3" path="/var/lib/kubelet/pods/abaafb18-9795-413a-a48e-381897e863f3/volumes" Jul 27 21:44:20.312982 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:20.312947 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll"] Jul 27 21:44:20.313510 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:20.313204 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" podUID="bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" containerName="istio-proxy" containerID="cri-o://4bcd0443a87004ea4d2118a1056cdeaeaf325149f2316589ae77ddee3434cd0d" gracePeriod=30 Jul 27 21:44:23.307392 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:23.307347 2565 patch_prober.go:28] interesting pod/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 27 21:44:23.307787 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:23.307415 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" podUID="bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 27 21:44:25.565432 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.565406 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:44:25.662835 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.662804 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-token\") pod \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " Jul 27 21:44:25.663023 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.662855 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-credential-socket\") pod \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " Jul 27 21:44:25.663023 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.662875 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-workload-certs\") pod \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " Jul 27 21:44:25.663023 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.662896 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istiod-ca-cert\") pod \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " Jul 27 21:44:25.663023 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.662950 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-envoy\") pod \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " Jul 27 21:44:25.663273 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.663122 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-workload-socket\") pod \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " Jul 27 21:44:25.663273 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.663155 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-data\") pod \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " Jul 27 21:44:25.663273 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.663140 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" (UID: "bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:44:25.663273 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.663192 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-podinfo\") pod \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " Jul 27 21:44:25.663273 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.663201 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" (UID: "bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:44:25.663528 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.663278 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmjlk\" (UniqueName: \"kubernetes.io/projected/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-kube-api-access-hmjlk\") pod \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\" (UID: \"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d\") " Jul 27 21:44:25.663528 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.663417 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" (UID: "bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:44:25.663528 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.663435 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" (UID: "bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 21:44:25.663768 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.663626 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-workload-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:44:25.663768 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.663642 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-credential-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:44:25.663768 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.663656 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-workload-certs\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:44:25.663768 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.663668 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istiod-ca-cert\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:44:25.664045 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.664006 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-data" (OuterVolumeSpecName: "istio-data") pod "bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" (UID: "bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:44:25.665307 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.665279 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-token" (OuterVolumeSpecName: "istio-token") pod "bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" (UID: "bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:44:25.665412 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.665347 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" (UID: "bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:44:25.665898 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.665872 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" (UID: "bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 27 21:44:25.665993 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.665876 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-kube-api-access-hmjlk" (OuterVolumeSpecName: "kube-api-access-hmjlk") pod "bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" (UID: "bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d"). InnerVolumeSpecName "kube-api-access-hmjlk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:44:25.764694 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.764658 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-envoy\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:44:25.764694 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.764689 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-data\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:44:25.764694 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.764699 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-podinfo\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:44:25.764939 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.764708 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmjlk\" (UniqueName: \"kubernetes.io/projected/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-kube-api-access-hmjlk\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:44:25.764939 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.764717 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d-istio-token\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:44:25.780313 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.780279 2565 generic.go:356] "Generic (PLEG): container finished" podID="bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" containerID="4bcd0443a87004ea4d2118a1056cdeaeaf325149f2316589ae77ddee3434cd0d" exitCode=0 Jul 27 21:44:25.780467 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.780327 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" Jul 27 21:44:25.780467 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.780355 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" event={"ID":"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d","Type":"ContainerDied","Data":"4bcd0443a87004ea4d2118a1056cdeaeaf325149f2316589ae77ddee3434cd0d"} Jul 27 21:44:25.780467 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.780406 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll" event={"ID":"bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d","Type":"ContainerDied","Data":"abac92e1b03d1effff196c48d08431bcecf124bec0521c65c9fe28ff1e110b92"} Jul 27 21:44:25.780467 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.780429 2565 scope.go:117] "RemoveContainer" containerID="4bcd0443a87004ea4d2118a1056cdeaeaf325149f2316589ae77ddee3434cd0d" Jul 27 21:44:25.789952 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.789933 2565 scope.go:117] "RemoveContainer" containerID="5cdf00c6ea91906160b165686b642c2490832eb9dd25f8f087ef522bbdf29f02" Jul 27 21:44:25.798459 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.798440 2565 scope.go:117] "RemoveContainer" containerID="4bcd0443a87004ea4d2118a1056cdeaeaf325149f2316589ae77ddee3434cd0d" Jul 27 21:44:25.798736 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:44:25.798720 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bcd0443a87004ea4d2118a1056cdeaeaf325149f2316589ae77ddee3434cd0d\": container with ID starting with 4bcd0443a87004ea4d2118a1056cdeaeaf325149f2316589ae77ddee3434cd0d not found: ID does not exist" containerID="4bcd0443a87004ea4d2118a1056cdeaeaf325149f2316589ae77ddee3434cd0d" Jul 27 21:44:25.798779 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.798743 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bcd0443a87004ea4d2118a1056cdeaeaf325149f2316589ae77ddee3434cd0d"} err="failed to get container status \"4bcd0443a87004ea4d2118a1056cdeaeaf325149f2316589ae77ddee3434cd0d\": rpc error: code = NotFound desc = could not find container \"4bcd0443a87004ea4d2118a1056cdeaeaf325149f2316589ae77ddee3434cd0d\": container with ID starting with 4bcd0443a87004ea4d2118a1056cdeaeaf325149f2316589ae77ddee3434cd0d not found: ID does not exist" Jul 27 21:44:25.798779 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.798759 2565 scope.go:117] "RemoveContainer" containerID="5cdf00c6ea91906160b165686b642c2490832eb9dd25f8f087ef522bbdf29f02" Jul 27 21:44:25.798974 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:44:25.798957 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cdf00c6ea91906160b165686b642c2490832eb9dd25f8f087ef522bbdf29f02\": container with ID starting with 5cdf00c6ea91906160b165686b642c2490832eb9dd25f8f087ef522bbdf29f02 not found: ID does not exist" containerID="5cdf00c6ea91906160b165686b642c2490832eb9dd25f8f087ef522bbdf29f02" Jul 27 21:44:25.799009 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.798981 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cdf00c6ea91906160b165686b642c2490832eb9dd25f8f087ef522bbdf29f02"} err="failed to get container status \"5cdf00c6ea91906160b165686b642c2490832eb9dd25f8f087ef522bbdf29f02\": rpc error: code = NotFound desc = could not find container \"5cdf00c6ea91906160b165686b642c2490832eb9dd25f8f087ef522bbdf29f02\": container with ID starting with 5cdf00c6ea91906160b165686b642c2490832eb9dd25f8f087ef522bbdf29f02 not found: ID does not exist" Jul 27 21:44:25.802913 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.802883 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll"] Jul 27 21:44:25.806874 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:25.806846 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-c910ce-openshift-default-c8dcd48f6-8fkll"] Jul 27 21:44:26.052292 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:26.052212 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" path="/var/lib/kubelet/pods/bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d/volumes" Jul 27 21:44:42.175423 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:42.175391 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm"] Jul 27 21:44:42.175834 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:42.175589 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" podUID="afc1af5c-6ffa-491c-9ffa-cebb5d217775" containerName="istio-proxy" containerID="cri-o://e7bec236d316e032a5b6d866c7ec799fa2e1a63ad319d6f6dbace3d29dae676e" gracePeriod=30 Jul 27 21:44:42.859020 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:42.858985 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8658494974-jpmxq_44a27f06-a463-48c6-85f3-a888e14dd522/authorino/0.log" Jul 27 21:44:44.866225 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:44.866191 2565 patch_prober.go:28] interesting pod/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 27 21:44:44.866623 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:44.866258 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" podUID="afc1af5c-6ffa-491c-9ffa-cebb5d217775" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 27 21:44:47.430648 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.430616 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:44:47.454751 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.454677 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-token\") pod \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " Jul 27 21:44:47.454751 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.454721 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-podinfo\") pod \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " Jul 27 21:44:47.454751 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.454749 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-envoy\") pod \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " Jul 27 21:44:47.455086 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.454780 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istiod-ca-cert\") pod \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " Jul 27 21:44:47.455086 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.454810 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-workload-certs\") pod \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " Jul 27 21:44:47.455086 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.454839 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-workload-socket\") pod \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " Jul 27 21:44:47.455086 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.454905 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tq8wh\" (UniqueName: \"kubernetes.io/projected/afc1af5c-6ffa-491c-9ffa-cebb5d217775-kube-api-access-tq8wh\") pod \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " Jul 27 21:44:47.455086 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.454937 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-credential-socket\") pod \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " Jul 27 21:44:47.455086 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.454981 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-data\") pod \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\" (UID: \"afc1af5c-6ffa-491c-9ffa-cebb5d217775\") " Jul 27 21:44:47.455711 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.455416 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "afc1af5c-6ffa-491c-9ffa-cebb5d217775" (UID: "afc1af5c-6ffa-491c-9ffa-cebb5d217775"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:44:47.455711 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.455486 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "afc1af5c-6ffa-491c-9ffa-cebb5d217775" (UID: "afc1af5c-6ffa-491c-9ffa-cebb5d217775"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:44:47.455711 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.455677 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "afc1af5c-6ffa-491c-9ffa-cebb5d217775" (UID: "afc1af5c-6ffa-491c-9ffa-cebb5d217775"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 21:44:47.455922 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.455713 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "afc1af5c-6ffa-491c-9ffa-cebb5d217775" (UID: "afc1af5c-6ffa-491c-9ffa-cebb5d217775"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:44:47.456297 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.456273 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-data" (OuterVolumeSpecName: "istio-data") pod "afc1af5c-6ffa-491c-9ffa-cebb5d217775" (UID: "afc1af5c-6ffa-491c-9ffa-cebb5d217775"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:44:47.457127 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.457104 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-token" (OuterVolumeSpecName: "istio-token") pod "afc1af5c-6ffa-491c-9ffa-cebb5d217775" (UID: "afc1af5c-6ffa-491c-9ffa-cebb5d217775"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:44:47.457539 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.457514 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afc1af5c-6ffa-491c-9ffa-cebb5d217775-kube-api-access-tq8wh" (OuterVolumeSpecName: "kube-api-access-tq8wh") pod "afc1af5c-6ffa-491c-9ffa-cebb5d217775" (UID: "afc1af5c-6ffa-491c-9ffa-cebb5d217775"). InnerVolumeSpecName "kube-api-access-tq8wh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 21:44:47.457633 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.457619 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "afc1af5c-6ffa-491c-9ffa-cebb5d217775" (UID: "afc1af5c-6ffa-491c-9ffa-cebb5d217775"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 27 21:44:47.457686 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.457634 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "afc1af5c-6ffa-491c-9ffa-cebb5d217775" (UID: "afc1af5c-6ffa-491c-9ffa-cebb5d217775"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 21:44:47.556367 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.556318 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tq8wh\" (UniqueName: \"kubernetes.io/projected/afc1af5c-6ffa-491c-9ffa-cebb5d217775-kube-api-access-tq8wh\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:44:47.556367 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.556361 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-credential-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:44:47.556367 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.556376 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-data\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:44:47.556617 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.556388 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-token\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:44:47.556617 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.556401 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-podinfo\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:44:47.556617 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.556411 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istio-envoy\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:44:47.556617 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.556421 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afc1af5c-6ffa-491c-9ffa-cebb5d217775-istiod-ca-cert\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:44:47.556617 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.556431 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-workload-certs\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:44:47.556617 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.556443 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afc1af5c-6ffa-491c-9ffa-cebb5d217775-workload-socket\") on node \"ip-10-0-131-232.ec2.internal\" DevicePath \"\"" Jul 27 21:44:47.893684 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.893591 2565 generic.go:356] "Generic (PLEG): container finished" podID="afc1af5c-6ffa-491c-9ffa-cebb5d217775" containerID="e7bec236d316e032a5b6d866c7ec799fa2e1a63ad319d6f6dbace3d29dae676e" exitCode=0 Jul 27 21:44:47.893684 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.893642 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" Jul 27 21:44:47.893862 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.893648 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" event={"ID":"afc1af5c-6ffa-491c-9ffa-cebb5d217775","Type":"ContainerDied","Data":"e7bec236d316e032a5b6d866c7ec799fa2e1a63ad319d6f6dbace3d29dae676e"} Jul 27 21:44:47.893862 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.893752 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm" event={"ID":"afc1af5c-6ffa-491c-9ffa-cebb5d217775","Type":"ContainerDied","Data":"c222cccb38de11acaf254715eda283d19c12ee15e80e9d78f3f1ed19dfcbfcdc"} Jul 27 21:44:47.893862 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.893774 2565 scope.go:117] "RemoveContainer" containerID="e7bec236d316e032a5b6d866c7ec799fa2e1a63ad319d6f6dbace3d29dae676e" Jul 27 21:44:47.903080 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.903060 2565 scope.go:117] "RemoveContainer" containerID="0bab1f508bd05cd8df1fd576170270bdc37c2481bf082efb98535bc0b0302067" Jul 27 21:44:47.911693 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.911673 2565 scope.go:117] "RemoveContainer" containerID="e7bec236d316e032a5b6d866c7ec799fa2e1a63ad319d6f6dbace3d29dae676e" Jul 27 21:44:47.911978 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:44:47.911959 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7bec236d316e032a5b6d866c7ec799fa2e1a63ad319d6f6dbace3d29dae676e\": container with ID starting with e7bec236d316e032a5b6d866c7ec799fa2e1a63ad319d6f6dbace3d29dae676e not found: ID does not exist" containerID="e7bec236d316e032a5b6d866c7ec799fa2e1a63ad319d6f6dbace3d29dae676e" Jul 27 21:44:47.912055 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.911991 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7bec236d316e032a5b6d866c7ec799fa2e1a63ad319d6f6dbace3d29dae676e"} err="failed to get container status \"e7bec236d316e032a5b6d866c7ec799fa2e1a63ad319d6f6dbace3d29dae676e\": rpc error: code = NotFound desc = could not find container \"e7bec236d316e032a5b6d866c7ec799fa2e1a63ad319d6f6dbace3d29dae676e\": container with ID starting with e7bec236d316e032a5b6d866c7ec799fa2e1a63ad319d6f6dbace3d29dae676e not found: ID does not exist" Jul 27 21:44:47.912055 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.912017 2565 scope.go:117] "RemoveContainer" containerID="0bab1f508bd05cd8df1fd576170270bdc37c2481bf082efb98535bc0b0302067" Jul 27 21:44:47.912294 ip-10-0-131-232 kubenswrapper[2565]: E0727 21:44:47.912274 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bab1f508bd05cd8df1fd576170270bdc37c2481bf082efb98535bc0b0302067\": container with ID starting with 0bab1f508bd05cd8df1fd576170270bdc37c2481bf082efb98535bc0b0302067 not found: ID does not exist" containerID="0bab1f508bd05cd8df1fd576170270bdc37c2481bf082efb98535bc0b0302067" Jul 27 21:44:47.912343 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.912301 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bab1f508bd05cd8df1fd576170270bdc37c2481bf082efb98535bc0b0302067"} err="failed to get container status \"0bab1f508bd05cd8df1fd576170270bdc37c2481bf082efb98535bc0b0302067\": rpc error: code = NotFound desc = could not find container \"0bab1f508bd05cd8df1fd576170270bdc37c2481bf082efb98535bc0b0302067\": container with ID starting with 0bab1f508bd05cd8df1fd576170270bdc37c2481bf082efb98535bc0b0302067 not found: ID does not exist" Jul 27 21:44:47.915869 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.915847 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm"] Jul 27 21:44:47.921921 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.921896 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-85f004-openshift-default-6646696f6-5c4qm"] Jul 27 21:44:47.981883 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:47.981854 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-72bqn_b5268780-a4cb-41e2-9775-5a50c9d84f24/manager/0.log" Jul 27 21:44:48.051857 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:48.051826 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afc1af5c-6ffa-491c-9ffa-cebb5d217775" path="/var/lib/kubelet/pods/afc1af5c-6ffa-491c-9ffa-cebb5d217775/volumes" Jul 27 21:44:48.103460 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:48.103425 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-55784848bf-82r7n_06301a5d-f175-4d9e-99d7-056173c19ae5/manager/0.log" Jul 27 21:44:48.659884 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:48.659855 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk_8d30f9a9-b61d-4410-ac5c-9d41441e2ff0/copy-manifests/0.log" Jul 27 21:44:48.666396 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:48.666374 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk_8d30f9a9-b61d-4410-ac5c-9d41441e2ff0/manager/0.log" Jul 27 21:44:50.027128 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:50.027103 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th_c027e754-ca8e-4a2b-8ca1-60677abd3f1b/util/0.log" Jul 27 21:44:50.033740 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:50.033714 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th_c027e754-ca8e-4a2b-8ca1-60677abd3f1b/pull/0.log" Jul 27 21:44:50.039590 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:50.039568 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th_c027e754-ca8e-4a2b-8ca1-60677abd3f1b/extract/0.log" Jul 27 21:44:50.272343 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:50.272312 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf_8b45875d-c575-4fb8-985d-ae556d78565e/util/0.log" Jul 27 21:44:50.278762 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:50.278693 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf_8b45875d-c575-4fb8-985d-ae556d78565e/pull/0.log" Jul 27 21:44:50.284569 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:50.284549 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf_8b45875d-c575-4fb8-985d-ae556d78565e/extract/0.log" Jul 27 21:44:50.522114 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:50.522077 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8658494974-jpmxq_44a27f06-a463-48c6-85f3-a888e14dd522/authorino/0.log" Jul 27 21:44:50.634782 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:50.634685 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-khcmr_580c2387-4c06-46cc-bb8c-38ddb73724bc/manager/0.log" Jul 27 21:44:50.839691 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:50.839661 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-cz96d_508970d0-2590-4bd5-bfc4-74cc2dc5013a/kuadrant-console-plugin/0.log" Jul 27 21:44:50.953703 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:50.953675 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6xgc4_07577e4f-3f3e-4399-b803-823cc453a561/registry-server/0.log" Jul 27 21:44:51.599707 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:51.599670 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5_f30bbd8c-d47c-442f-8fd8-87350cd7ea03/istio-proxy/0.log" Jul 27 21:44:52.045189 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:52.045140 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-pt9n5_2647a034-e168-465b-aad6-8f7a13f91580/istio-proxy/1.log" Jul 27 21:44:52.807742 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:52.807710 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-t2twp_489e7645-73f9-4487-b3bb-8fe2d316ff02/main/0.log" Jul 27 21:44:53.026128 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:44:53.026098 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9sbld_788b3609-5f93-4615-9d60-1d87132a8836/main/0.log" Jul 27 21:45:00.143397 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.143355 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753145-9tmv2"] Jul 27 21:45:00.143881 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.143831 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" containerName="istio-proxy" Jul 27 21:45:00.143881 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.143849 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" containerName="istio-proxy" Jul 27 21:45:00.143881 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.143863 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abaafb18-9795-413a-a48e-381897e863f3" containerName="istio-proxy" Jul 27 21:45:00.143881 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.143871 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="abaafb18-9795-413a-a48e-381897e863f3" containerName="istio-proxy" Jul 27 21:45:00.144088 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.143884 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afc1af5c-6ffa-491c-9ffa-cebb5d217775" containerName="istio-proxy" Jul 27 21:45:00.144088 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.143894 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="afc1af5c-6ffa-491c-9ffa-cebb5d217775" containerName="istio-proxy" Jul 27 21:45:00.144088 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.143905 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dbda4df-3439-4e59-a37c-1dd761819778" containerName="istio-proxy" Jul 27 21:45:00.144088 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.143913 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dbda4df-3439-4e59-a37c-1dd761819778" containerName="istio-proxy" Jul 27 21:45:00.144088 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.143926 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abaafb18-9795-413a-a48e-381897e863f3" containerName="istio-proxy" Jul 27 21:45:00.144088 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.143935 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="abaafb18-9795-413a-a48e-381897e863f3" containerName="istio-proxy" Jul 27 21:45:00.144088 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.143949 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dbda4df-3439-4e59-a37c-1dd761819778" containerName="istio-proxy" Jul 27 21:45:00.144088 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.143957 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dbda4df-3439-4e59-a37c-1dd761819778" containerName="istio-proxy" Jul 27 21:45:00.144088 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.143967 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" containerName="istio-proxy" Jul 27 21:45:00.144088 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.143975 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" containerName="istio-proxy" Jul 27 21:45:00.144088 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.144076 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="abaafb18-9795-413a-a48e-381897e863f3" containerName="istio-proxy" Jul 27 21:45:00.144088 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.144087 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dbda4df-3439-4e59-a37c-1dd761819778" containerName="istio-proxy" Jul 27 21:45:00.144663 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.144097 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="afc1af5c-6ffa-491c-9ffa-cebb5d217775" containerName="istio-proxy" Jul 27 21:45:00.144663 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.144106 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="afc1af5c-6ffa-491c-9ffa-cebb5d217775" containerName="istio-proxy" Jul 27 21:45:00.144663 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.144116 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" containerName="istio-proxy" Jul 27 21:45:00.144663 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.144129 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfaa5a15-c1e7-4ab4-bf7e-5ead0e6c819d" containerName="istio-proxy" Jul 27 21:45:00.147171 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.147148 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753145-9tmv2" Jul 27 21:45:00.149921 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.149897 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4fvvt\"" Jul 27 21:45:00.155154 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.155129 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753145-9tmv2"] Jul 27 21:45:00.158183 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.158162 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753145-9tmv2" Jul 27 21:45:00.304730 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.304696 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753145-9tmv2"] Jul 27 21:45:00.307014 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:45:00.306972 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a9f85fc_2499_4a74_900a_1d45dce67955.slice/crio-ee571a9ee84c322f21f01582f273b3c0b4ac6a6cc459ebf4fb941826b8f288ee WatchSource:0}: Error finding container ee571a9ee84c322f21f01582f273b3c0b4ac6a6cc459ebf4fb941826b8f288ee: Status 404 returned error can't find the container with id ee571a9ee84c322f21f01582f273b3c0b4ac6a6cc459ebf4fb941826b8f288ee Jul 27 21:45:00.309251 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.309218 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 21:45:00.959388 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:00.959351 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753145-9tmv2" event={"ID":"0a9f85fc-2499-4a74-900a-1d45dce67955","Type":"ContainerStarted","Data":"ee571a9ee84c322f21f01582f273b3c0b4ac6a6cc459ebf4fb941826b8f288ee"} Jul 27 21:45:01.616032 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:01.615996 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qzsr7_86fb8fed-0e82-44c5-9c08-08a9b4d8eea3/global-pull-secret-syncer/0.log" Jul 27 21:45:01.717304 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:01.717267 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-td7kp_e8a1339c-f7dc-4b4e-9a09-c82402afe736/konnectivity-agent/0.log" Jul 27 21:45:01.770493 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:01.770458 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-232.ec2.internal_9233b835deeeffbb2950b8b4fe9f0d75/haproxy/0.log" Jul 27 21:45:03.975150 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:03.975110 2565 generic.go:356] "Generic (PLEG): container finished" podID="0a9f85fc-2499-4a74-900a-1d45dce67955" containerID="04fa3c0f3c5e99865d0297cc023df6881c6f357c3144f411537e93a5910dfbe6" exitCode=0 Jul 27 21:45:03.975552 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:03.975201 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753145-9tmv2" event={"ID":"0a9f85fc-2499-4a74-900a-1d45dce67955","Type":"ContainerDied","Data":"04fa3c0f3c5e99865d0297cc023df6881c6f357c3144f411537e93a5910dfbe6"} Jul 27 21:45:05.117362 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:05.117334 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753145-9tmv2" Jul 27 21:45:05.987548 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:05.987505 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753145-9tmv2" event={"ID":"0a9f85fc-2499-4a74-900a-1d45dce67955","Type":"ContainerDied","Data":"ee571a9ee84c322f21f01582f273b3c0b4ac6a6cc459ebf4fb941826b8f288ee"} Jul 27 21:45:05.987548 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:05.987543 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753145-9tmv2" Jul 27 21:45:05.987760 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:05.987561 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee571a9ee84c322f21f01582f273b3c0b4ac6a6cc459ebf4fb941826b8f288ee" Jul 27 21:45:06.492732 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:06.492694 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th_c027e754-ca8e-4a2b-8ca1-60677abd3f1b/extract/0.log" Jul 27 21:45:06.512762 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:06.512737 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th_c027e754-ca8e-4a2b-8ca1-60677abd3f1b/util/0.log" Jul 27 21:45:06.533068 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:06.533024 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599d2th_c027e754-ca8e-4a2b-8ca1-60677abd3f1b/pull/0.log" Jul 27 21:45:06.625960 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:06.625932 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf_8b45875d-c575-4fb8-985d-ae556d78565e/extract/0.log" Jul 27 21:45:06.647645 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:06.647614 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf_8b45875d-c575-4fb8-985d-ae556d78565e/util/0.log" Jul 27 21:45:06.668434 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:06.668403 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739xczf_8b45875d-c575-4fb8-985d-ae556d78565e/pull/0.log" Jul 27 21:45:07.350995 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:07.350958 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8658494974-jpmxq_44a27f06-a463-48c6-85f3-a888e14dd522/authorino/0.log" Jul 27 21:45:07.436478 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:07.436443 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-khcmr_580c2387-4c06-46cc-bb8c-38ddb73724bc/manager/0.log" Jul 27 21:45:07.493978 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:07.493950 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-cz96d_508970d0-2590-4bd5-bfc4-74cc2dc5013a/kuadrant-console-plugin/0.log" Jul 27 21:45:07.551293 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:07.551263 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6xgc4_07577e4f-3f3e-4399-b803-823cc453a561/registry-server/0.log" Jul 27 21:45:09.888216 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:09.888183 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lvtjr_11ab4a48-f264-4bb7-86ec-778b13da8825/node-exporter/0.log" Jul 27 21:45:09.906939 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:09.906895 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lvtjr_11ab4a48-f264-4bb7-86ec-778b13da8825/kube-rbac-proxy/0.log" Jul 27 21:45:09.925488 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:09.925461 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lvtjr_11ab4a48-f264-4bb7-86ec-778b13da8825/init-textfile/0.log" Jul 27 21:45:10.325065 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:10.325038 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6bdd54d9bb-kjr8s_aac785a0-88db-4144-ac71-bdd8b14388a5/telemeter-client/0.log" Jul 27 21:45:10.352325 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:10.352291 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6bdd54d9bb-kjr8s_aac785a0-88db-4144-ac71-bdd8b14388a5/reload/0.log" Jul 27 21:45:10.375344 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:10.375313 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6bdd54d9bb-kjr8s_aac785a0-88db-4144-ac71-bdd8b14388a5/kube-rbac-proxy/0.log" Jul 27 21:45:11.442480 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.442444 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns"] Jul 27 21:45:11.442888 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.442822 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afc1af5c-6ffa-491c-9ffa-cebb5d217775" containerName="istio-proxy" Jul 27 21:45:11.442888 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.442835 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="afc1af5c-6ffa-491c-9ffa-cebb5d217775" containerName="istio-proxy" Jul 27 21:45:11.442888 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.442847 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a9f85fc-2499-4a74-900a-1d45dce67955" containerName="cleanup" Jul 27 21:45:11.442888 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.442853 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a9f85fc-2499-4a74-900a-1d45dce67955" containerName="cleanup" Jul 27 21:45:11.443019 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.442939 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a9f85fc-2499-4a74-900a-1d45dce67955" containerName="cleanup" Jul 27 21:45:11.443019 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.442956 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="abaafb18-9795-413a-a48e-381897e863f3" containerName="istio-proxy" Jul 27 21:45:11.443019 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.442964 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dbda4df-3439-4e59-a37c-1dd761819778" containerName="istio-proxy" Jul 27 21:45:11.446053 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.446027 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" Jul 27 21:45:11.450039 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.450020 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-d7w9b\"/\"default-dockercfg-vqvld\"" Jul 27 21:45:11.450264 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.450248 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d7w9b\"/\"kube-root-ca.crt\"" Jul 27 21:45:11.451323 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.451303 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d7w9b\"/\"openshift-service-ca.crt\"" Jul 27 21:45:11.458171 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.458149 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns"] Jul 27 21:45:11.563347 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.563308 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7135f163-13ae-4ba3-9622-182642698905-proc\") pod \"perf-node-gather-daemonset-mhtns\" (UID: \"7135f163-13ae-4ba3-9622-182642698905\") " pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" Jul 27 21:45:11.563525 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.563376 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h6hz\" (UniqueName: \"kubernetes.io/projected/7135f163-13ae-4ba3-9622-182642698905-kube-api-access-5h6hz\") pod \"perf-node-gather-daemonset-mhtns\" (UID: \"7135f163-13ae-4ba3-9622-182642698905\") " pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" Jul 27 21:45:11.563525 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.563448 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7135f163-13ae-4ba3-9622-182642698905-podres\") pod \"perf-node-gather-daemonset-mhtns\" (UID: \"7135f163-13ae-4ba3-9622-182642698905\") " pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" Jul 27 21:45:11.563525 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.563499 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7135f163-13ae-4ba3-9622-182642698905-lib-modules\") pod \"perf-node-gather-daemonset-mhtns\" (UID: \"7135f163-13ae-4ba3-9622-182642698905\") " pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" Jul 27 21:45:11.563637 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.563527 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7135f163-13ae-4ba3-9622-182642698905-sys\") pod \"perf-node-gather-daemonset-mhtns\" (UID: \"7135f163-13ae-4ba3-9622-182642698905\") " pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" Jul 27 21:45:11.664529 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.664490 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7135f163-13ae-4ba3-9622-182642698905-proc\") pod \"perf-node-gather-daemonset-mhtns\" (UID: \"7135f163-13ae-4ba3-9622-182642698905\") " pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" Jul 27 21:45:11.664692 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.664553 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h6hz\" (UniqueName: \"kubernetes.io/projected/7135f163-13ae-4ba3-9622-182642698905-kube-api-access-5h6hz\") pod \"perf-node-gather-daemonset-mhtns\" (UID: \"7135f163-13ae-4ba3-9622-182642698905\") " pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" Jul 27 21:45:11.664692 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.664573 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7135f163-13ae-4ba3-9622-182642698905-podres\") pod \"perf-node-gather-daemonset-mhtns\" (UID: \"7135f163-13ae-4ba3-9622-182642698905\") " pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" Jul 27 21:45:11.664692 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.664601 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7135f163-13ae-4ba3-9622-182642698905-lib-modules\") pod \"perf-node-gather-daemonset-mhtns\" (UID: \"7135f163-13ae-4ba3-9622-182642698905\") " pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" Jul 27 21:45:11.664692 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.664622 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7135f163-13ae-4ba3-9622-182642698905-sys\") pod \"perf-node-gather-daemonset-mhtns\" (UID: \"7135f163-13ae-4ba3-9622-182642698905\") " pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" Jul 27 21:45:11.664692 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.664618 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7135f163-13ae-4ba3-9622-182642698905-proc\") pod \"perf-node-gather-daemonset-mhtns\" (UID: \"7135f163-13ae-4ba3-9622-182642698905\") " pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" Jul 27 21:45:11.664854 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.664696 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7135f163-13ae-4ba3-9622-182642698905-sys\") pod \"perf-node-gather-daemonset-mhtns\" (UID: \"7135f163-13ae-4ba3-9622-182642698905\") " pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" Jul 27 21:45:11.664854 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.664729 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7135f163-13ae-4ba3-9622-182642698905-lib-modules\") pod \"perf-node-gather-daemonset-mhtns\" (UID: \"7135f163-13ae-4ba3-9622-182642698905\") " pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" Jul 27 21:45:11.664854 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.664734 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7135f163-13ae-4ba3-9622-182642698905-podres\") pod \"perf-node-gather-daemonset-mhtns\" (UID: \"7135f163-13ae-4ba3-9622-182642698905\") " pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" Jul 27 21:45:11.678984 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.678944 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h6hz\" (UniqueName: \"kubernetes.io/projected/7135f163-13ae-4ba3-9622-182642698905-kube-api-access-5h6hz\") pod \"perf-node-gather-daemonset-mhtns\" (UID: \"7135f163-13ae-4ba3-9622-182642698905\") " pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" Jul 27 21:45:11.704414 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.704335 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-9h98q_9736f967-2f63-40e7-966e-628f22fe91d6/networking-console-plugin/0.log" Jul 27 21:45:11.755791 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.755751 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" Jul 27 21:45:11.888219 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:11.888187 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns"] Jul 27 21:45:11.889498 ip-10-0-131-232 kubenswrapper[2565]: W0727 21:45:11.889471 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7135f163_13ae_4ba3_9622_182642698905.slice/crio-cd99e96f04c6406494ed81ac4be3d2ad2d9454fa43c6011568df8395e1b51a89 WatchSource:0}: Error finding container cd99e96f04c6406494ed81ac4be3d2ad2d9454fa43c6011568df8395e1b51a89: Status 404 returned error can't find the container with id cd99e96f04c6406494ed81ac4be3d2ad2d9454fa43c6011568df8395e1b51a89 Jul 27 21:45:12.018862 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:12.018825 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" event={"ID":"7135f163-13ae-4ba3-9622-182642698905","Type":"ContainerStarted","Data":"58367a64695ccf9b8f7e16430578a71cf91dcd57bbc8d2eb640643934afb866c"} Jul 27 21:45:12.019033 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:12.018884 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" event={"ID":"7135f163-13ae-4ba3-9622-182642698905","Type":"ContainerStarted","Data":"cd99e96f04c6406494ed81ac4be3d2ad2d9454fa43c6011568df8395e1b51a89"} Jul 27 21:45:12.019033 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:12.018965 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" Jul 27 21:45:12.036358 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:12.036299 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" podStartSLOduration=1.036283181 podStartE2EDuration="1.036283181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:45:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:45:12.034604093 +0000 UTC m=+3240.557721829" watchObservedRunningTime="2026-07-27 21:45:12.036283181 +0000 UTC m=+3240.559400902" Jul 27 21:45:13.361128 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:13.361094 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-k67sh_8633760a-7d53-4b16-bbf0-401f9fa8b8ec/volume-data-source-validator/0.log" Jul 27 21:45:14.446324 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:14.446286 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-j2k9d_c1f55707-7360-492b-a75f-f2211a512fd3/dns/0.log" Jul 27 21:45:14.463385 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:14.463355 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-j2k9d_c1f55707-7360-492b-a75f-f2211a512fd3/kube-rbac-proxy/0.log" Jul 27 21:45:14.511761 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:14.511728 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lbt9p_78a12024-7c5c-4f1c-a15d-58876f88ee45/dns-node-resolver/0.log" Jul 27 21:45:15.136854 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:15.136820 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-74fh5_3705087b-83f8-4e75-84e2-ce01936e4657/node-ca/0.log" Jul 27 21:45:16.152742 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:16.152713 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5_f30bbd8c-d47c-442f-8fd8-87350cd7ea03/istio-proxy/0.log" Jul 27 21:45:16.397303 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:16.397267 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-pt9n5_2647a034-e168-465b-aad6-8f7a13f91580/istio-proxy/0.log" Jul 27 21:45:16.446255 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:16.446196 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-pt9n5_2647a034-e168-465b-aad6-8f7a13f91580/istio-proxy/1.log" Jul 27 21:45:17.403875 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:17.403843 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5psdr_78365e64-dcf0-450f-ae1e-d8dc161e3657/serve-healthcheck-canary/0.log" Jul 27 21:45:17.914592 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:17.914560 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4bhvm_f7db41ad-fec7-4420-93d8-869d4ee1965b/kube-rbac-proxy/0.log" Jul 27 21:45:17.934513 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:17.934482 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4bhvm_f7db41ad-fec7-4420-93d8-869d4ee1965b/exporter/0.log" Jul 27 21:45:17.952617 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:17.952582 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4bhvm_f7db41ad-fec7-4420-93d8-869d4ee1965b/extractor/0.log" Jul 27 21:45:18.035364 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:18.035338 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-d7w9b/perf-node-gather-daemonset-mhtns" Jul 27 21:45:20.758841 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:20.758810 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-72bqn_b5268780-a4cb-41e2-9775-5a50c9d84f24/manager/0.log" Jul 27 21:45:21.025676 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:21.025634 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-55784848bf-82r7n_06301a5d-f175-4d9e-99d7-056173c19ae5/manager/0.log" Jul 27 21:45:21.305312 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:21.305284 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk_8d30f9a9-b61d-4410-ac5c-9d41441e2ff0/manager/0.log" Jul 27 21:45:21.323842 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:21.323818 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b4c7b8d87-wn5qk_8d30f9a9-b61d-4410-ac5c-9d41441e2ff0/copy-manifests/0.log" Jul 27 21:45:21.846782 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:21.846675 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5_f30bbd8c-d47c-442f-8fd8-87350cd7ea03/istio-proxy/0.log" Jul 27 21:45:22.037870 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:22.037838 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-pt9n5_2647a034-e168-465b-aad6-8f7a13f91580/istio-proxy/0.log" Jul 27 21:45:22.089915 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:22.089882 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-pt9n5_2647a034-e168-465b-aad6-8f7a13f91580/istio-proxy/1.log" Jul 27 21:45:22.557723 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:22.557692 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-58799fcf76-s22kw_9253d899-26f8-431e-8bb3-311580627a62/manager/0.log" Jul 27 21:45:22.607987 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:22.607952 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-cmhhb_4b7d257f-94bb-4b12-9cdc-7344915e51d7/openshift-lws-operator/0.log" Jul 27 21:45:23.567168 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:23.567130 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qfhd5_f30bbd8c-d47c-442f-8fd8-87350cd7ea03/istio-proxy/0.log" Jul 27 21:45:23.769444 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:23.769412 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-pt9n5_2647a034-e168-465b-aad6-8f7a13f91580/istio-proxy/0.log" Jul 27 21:45:23.821547 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:23.821453 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-pt9n5_2647a034-e168-465b-aad6-8f7a13f91580/istio-proxy/1.log" Jul 27 21:45:28.756681 ip-10-0-131-232 kubenswrapper[2565]: I0727 21:45:28.756649 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6vdnr_db168b68-5746-4d6a-b2ee-fc30b50bf424/kube-multus/0.log"